English French German Italian Portuguese Russian Spanish
Tools Storytelling

Storytelling

Storytelling provides a powerful means to obtain information on a project’s outcomes from participants’ experiences and viewpoints. Storytelling provides meaningful information that can highlight both the strong points and weaknesses of a project, as well as any unintended consequences. In a way, by asking participants to provide a story on a project, it asks them to evaluate an aspect of a project, rather than provide information for someone else to place a value on, storytelling generally brings out memorable or momentous experiences.

Storytelling has a number of benefits including:

  • Understanding the project from the viewpoint of the participants
  • Reflects the importance of context and its impact on outcomes
  • Allows the identification of unintended consequences
  • Provides a means to engage participants in evaluation.


Excerpts from stories can be used alongside graphs or in reports to provide a first person voice as to a project’s impact.
The Most Significant Change is methodology to collect, analyse and evaluate stories that is increasingly being used in evaluating projects both in Australia and overseas.


Stories can be collected through blog postings on a project’s website. For example, the Zero Carbon Moreland project have a webpage of participants’ stories.


Appreciative Inquiry is a form of research that looks at investigating a project’s positive outcomes in order to understand and replicate these rather than looking to understand why a project has not worked.


The California Endowment’s Storytelling Approaches to Program Evaluation: An Introduction provides a guide to other storytelling approaches.

Storytelling approaches to program evaluation

THE MOST SIGNICANT CHANGE

The Most Significant Change (MSC) is a method of participatory evaluation that involves the collection of significant change stories at different levels of the intervention (for example project staff, change agents, intervention participants) and collectively deciding on the most significant change stories based on selected themes (called domains). The domains reflect broad categories, such as change in capacity to take action, or a change in participation in an activity, as well as lessons learned.

MSC has its roots in the overseas development field but is becoming more widely used in community development and behaviour change interventions in Australia.

Davies and Dart (2005) list the following benefits from using the MSC method:

  • It is a good means of identifying unexpected changes.
  • It is a good way to clearly identify the values that prevail in an organisation and to have a practical discussion about which of those values are the most important.
  • It is a participatory form of monitoring that requires no special professional skills. Compared to other monitoring approaches, it is easy to communicate across cultures. There is no need to explain what an indicator is. Everyone can tell stories about events they think were important.
  • It encourages analysis as well as data collection because people have to explain why they believe one change is more important than another.
  • It can build staff capacity in analysing data and conceptualising impact.
  • It can deliver a rich picture of what is happening, rather than an overly simplified picture where organisational, social and economic developments are reduced to a single number.
  • It can be used to monitor and evaluate bottom-up initiatives that do not have predefined outcomes against which to evaluate.

Pros and cons of MSC

Pros Cons
Participatory process of evaluation involving all stakeholders Requires all stakeholders to accept method
Focus on learning and process of change May require training in MSC
Can capture unintended consequence or emergence Time consuming

 

Requirements

  • A person to lead and coordinate the MSC process
  • People to collect stories. This can involve having one or two people collecting stories, or getting participants to record stories from one another. Alternatively, you could ask people to write their own stories.
  • Ways to record stories of change - digital voice recorder, or film (remember that you must seek all participant’s approval prior to recording) or notepaper/template. You can also complement this by photography or images of change, taken by the storytellers.
  • A document tracking sheet to record who has provided a story, and what stories are classified as most significant.

Process

The following provide a guide on using the MSC technique:

MSC Quick Start GuideThe Most Significant Change Technique User GuideAustralian Government Land and Coasts Developing a Performance Story Report
MSC Quick Start Guide MSC Complete Guide

Developing a performance story report

see p31

 

 

The main steps to design an MSC process

1. Starting This is where you should introduce the MSC process to all the stakeholders and clarify its purpose. It is important to also consider who are the best people involved in the project to collect stories from.
2. Establishing 'domains of change'

Domains are broad and often fuzzy categories of possible changes. For example, participants in the MSC could be asked to look for significant changes in four domains:

Changes in the quality of people's lives

Changes in the nature of people's participation in sustainability activities

Changes in the sustainability of people's lives, organisations and activities

Any other changes

3. Collecting Significant Change (SC) stories

The central part of MSC is an open question to participants, such as:‘Looking back over the last month, what do you think was the most significant change in the participants practice of sustainability in their home life?’

The question has six parts:

‘Looking back over the last month…’ – It refers to a specific time period.

‘…what do you think was...’ – It asks respondents to exercise their own judgment.

‘…the most significant…’ – It asks respondents to be selective, not to try to comment on everything, but to focus in and report on one thing.

‘…change…’ – It asks respondents to be more selective, to report a change rather than static aspects of the situation or something that was present in the previous reporting period.

‘…in the participants practice of sustainability …’ – It asks respondents to be even more selective, not to report just any change but a change in the participants understanding of sustainability. This tag describes a domain of change and can be modified to fit other domains of change (for example, participants understanding of sustainability in the community).

‘…in their home life?’ – Like the first part of the sentence, this establishes some boundaries. In this particular case we are not asking about people’s actions in the community or what others are doing in Australia, but focussing about people’s home life.

A template to collect stories, as used in the Sustainable Ambassador project, is available here.

4. Determining who will review the SC stories Selecting stories usually involves a hierarchical process, where the lower levels select significant stories for the upper levels to review, and select the most significant ones from the lower levels.
5. Determining a process for reviewing the SC stories A template to assign domains and select the most significant stories is available here.
6. Feedback It is really important to feed back your selected SC stories and the reasons for the choice to the relevant people.

 

Examples of the use of MSC

The Castlemaine 500 project used the MSC method to complement quantitative data collected through billing data.  The MSC method showed a community-wide excitement around the topic of energy efficiency, and that a capacity to change had been developed in the community, even though the quantitative savings were not great. This underscores the reason why it is important to balance quantitative data with qualitative information, especially for projects that involve people changing behaviours. Behaviour changes, unlike the models of behaviour suggest, is not a simple process, and occurs through time, and amongst other supporting and conflicting influences and factors. In the case of the Castlemaine 500 project, if evaluation had solely rested on quantitative methods, the project could be construed as not having met its goals, whereas the use of qualitative data shows otherwise.

Case study on MSC

The Ambassadors’ Stories- a case study on using the Most Significant Change method with the Whitehorse Sustainable Ambassadors

Further  Links & Resources

Rick Davies MSC site
The most significant lessons about the Most Significant Change technique
MSC resources (including links to powerpoints) from the Department of Education and Children’s Services (SA)

 
Donate Now

Please make a donation to upgrade the Evaluation Toolbox.

The Evaluation Toolbox is maintained by Damien Sweeney and Martin Pritchard from PREA as their in-kind contribution back to the community. The Toolbox now needs several weeks of work to review and upgrade all the contents and add new content for you. Work has begun and we are seeking your donation (big or small) to help support this major upgrade. Email us to indicate what you want to be included in the Toolbox.

case study

What is the Toolbox?

The toolbox aims to provide a one-stop-site for the evaluation of community sustainability engagement projects that aim to change household behaviours. Through the toolbox you can learn how to conduct your own evaluation of a behaviour change project using the guides and templates provided.

see more...

Why Evaluate?

Conducting an evaluation is considered good practice in managing a project.

The monitoring phase of project evaluation allows us to track progress and identify issues early during implementation, thus providing and opportunity to take corrective action or make proactive improvements as required.

see more...

 

DSE
City of Whitehorse City of Whitehorse
Accord