English French German Italian Portuguese Russian Spanish
Planning Your Evaluation What is evaluation

What is Evaluation

There are many definitions of evaluation in the literature and websites. For the purpose of this guide, we will define evaluation as a structured process of assessing the success of a project in meeting its goals and to reflect on the lessons learned.

Project

A project typically relates to a set of specific activities within a set timeline.

Program

A program typically has a broader scope, and can consist of several ongoing projects within a broader timeframe.

An evaluation should be structured so that there is some thought and intent as to what is to be captured, how best to capture it, and what the analysis of the captured data will tell us about the project.

Another term that is widely used is monitoring. Monitoring refers to setting targets and milestones to measure progress and achievement, and whether the inputs are producing the planned outputs. In other words, monitoring sees whether the project is consistent with the design.

The key difference between monitoring and evaluation is that evaluation is about placing a value judgement on the information gathered during a project, including the monitoring data. The assessment of a project’s success (its evaluation) can be different based on whose value judgement is used. For example, a project manager’s evaluation may be different to that of the project’s participants, or other stakeholders.

A short tutorial on evaluation- opens up in PowerPoint

What is Evaluation presentation front slide

 

Evaluation Terminology

The language and terms used in evaluation can make the whole process quite daunting. This is accentuated by many references providing different definitions for the same term. The important thing for you to do is not to get bogged down in all the jargon, but to make sure you use the same terms consistently within your evaluation. It may help to provide a brief definition of the terms you select in your evaluation report, so that readers know what you mean when you use words that may have different meanings.

Some Evaluation Terminology to Consider

Evaluation Term
Definition

Activities

The tasks that are required to be done in order to achieve project outputs (eg. run a workshop, conduct and audit)

Efficiency

Refers to the extent to which activities, outputs and/or the desired effects are achieved with the lowest possible use of resources/inputs (funds, expertise, time)

Effectiveness

The extent to which project meets its intended outputs and/or objectives.

Impact

Refers to the measures of change that result from the outputs being completed, such as responses to surveys, requests for further information, or number of products taken up (eg. lights installed).


Impact is sometimes used in place of short-term outcomes

Qualitative

Refers to data that consists of words, or communication (whether that is text, voice, or visual).

Quantitative

Refers to data that are counts or numbers.

Outcome

Measures the change in behaviour or resource use in relation to goal of the project. Outcomes are usually considered in terms of their expected timeframe:

· Short–term (or immediate),

· Intermediate, and

· Long-term.

Without thorough outcome evaluation, it is not possible to demonstrate whether a behaviour change project has had the desired effect. It is important to capture both intended and unintended outcomes.

Outputs

Products or services delivered as part of the project’s activities (eg. workshops, audits, brochures).

Relevance

The extent to which the project purpose and goal meet the target group’s needs or priorities.

Sustainability

In terms of a project, sustainability refers to the likelihood of the change continuing once the intervention activities have ceased.

You will find different definitions in various other resources, so the important thing is to be consistent with what you choose to use.

 

For a list of further terms and definitions, see pp. 4-6 of Does Your Project Make a Difference.

Does your project make a difference

 

 


SPLASH & RIPPLES- A way to visualise outcome measurement
From PLAN:NET LIMITED (2008) Splash & Ripple: Using Outcomes to Design & Manage Community Activities page 5

The rock is like a material input, the person holding the rock is like a human resource input. The act of dropping the rock is like an activity. When the rock reaches the water, it creates a SPLASH. These are your outputs. The RIPPLES, spreading out from the splash are like your outcomes, (short, medium and long term). The edge of the pond represents the geographic and population boundaries of your project.

Splash & Ripple Effect

You can download A Short Guide to Monitoring and Evaluation as a reference document that provides a high level overview of the contents of this toolbox.

A Short Guide to Monitoring and Evaluation

 

Some further reading to keep you busy....Wink

 

Guide to using social research in sustainability projects

Evaluation step-by-step guide

Evaluating community engagement

Developing and using program logic

The effective engagement toolkit (See the full Effective Engagement series)

Tools and approaches for evaluating engagement

Handbook on planning, monitoring and evaluating for development results

W.K Kellogg Foundation Evaluation Handbook

Reflect and Improve: A Tool Kit for Engaging Youth and Adults as Partners in Program Evaluation

A developmental evaluation primer

A developmental evaluation primer

Splash & Ripples: usiing outcomes to design and manager community activities

Splash & Ripples: Using outcomes to design and manage community activities

Social Learning Through Evaluation

Social learning through evaluation- from evidence based management to collective action for complex problems

efs handbook

Education for Sustainability in Local Government Handbook

(See Part 8 - Evaluation of Education for Sustainability Programs)

 

 

 

 

 
Donate Now

Please make a donation to upgrade the Evaluation Toolbox.

The Evaluation Toolbox is maintained by Damien Sweeney and Martin Pritchard from PREA as their in-kind contribution back to the community. The Toolbox now needs several weeks of work to review and upgrade all the contents and add new content for you. Work has begun and we are seeking your donation (big or small) to help support this major upgrade. Email us to indicate what you want to be included in the Toolbox.

case study

What is the Toolbox?

The toolbox aims to provide a one-stop-site for the evaluation of community sustainability engagement projects that aim to change household behaviours. Through the toolbox you can learn how to conduct your own evaluation of a behaviour change project using the guides and templates provided.

see more...

Why Evaluate?

Conducting an evaluation is considered good practice in managing a project.

The monitoring phase of project evaluation allows us to track progress and identify issues early during implementation, thus providing and opportunity to take corrective action or make proactive improvements as required.

see more...

 

DSE
City of Whitehorse City of Whitehorse
Accord