INTRODUCTION TO EVALUATION AND GENERAL GUIDANCE
What is evaluation? Why do it? Is it a tool for improving programs? Does it feel intimidating and burdensome because you don’t know much about it? What is the state of play in evaluation of peacebuilding? This section provides introductory information on evaluation of peacebuilding and background on how peacebuilding evaluation has evolved.
Guide Contents
What Types of Evaluations are There?
Process Evaluation
-
The University of Texas at Austin, Center for Teaching and Learning. “Process Evaluation.”
BEGINNER
This short text defines what process evaluation is, and lists what are some of its potential uses and the questions it may address. Additionally, examples and other resources are also available.
-
Centers for Disease Control and Prevention (CDC). Developing Process Evaluation Questions, Evaluation Brief 4. CDC, 2009.
INTERMEDIATE
This brief defines process evaluation and helps think about process evaluation questions and the type of information that they can help provide. Additionally, the brief presents a series of steps to develop process evaluation questions and finishes with a series of examples of process evaluation questions.
Impact Evaluation
-
Chigas, Diana, Madeline Church, and Vanessa Corlazzoli. “Evaluating Impacts of Peacebuilding Interventions: Approaches and Methods, Challenges and Considerations.” CCVRI Guidance Series. London: DFID, 2014.
INTERMEDIATE
This guidance is intended to help staff a) understand the challenges and opportunities of evaluating impacts of interventions that aim to build and achieve peace by targeting, directly or indirectly, key drivers of conflict and peace (peacebuilding interventions), and b) work with evaluators to ensure evaluation designs are appropriate, robust and conflict-sensitive. See pages 2-10 for an introduction to impact evaluation and its purposes.
-
InterAction. Impact Evaluation Guidance Note and Webinar Series. Washington, D.C.: InterAction, 2012.
ADVANCEDINTERMEDIATE
This series contains a series of guidance notes (in English, French, Spanish and Arabic), webinars on approaches and methods, and webinars on NGO experiences with methods. There are four guidance notes with associated webinars:
- An Introduction to Impact Evaluation (including how it differs from other types of evaluation and why it should be done, when and by whom);
- Linking Monitoring and Evaluation to Impact Evaluation (illustrating the relationship between routine M&E and impact evaluation – in particular, how both monitoring and evaluation activities can support meaningful and valid impact evaluation)
- Introduction to Mixed Methods in Impact Evaluation (explaining) what a mixed methods impact evaluation design is and what distinguishes this approach from quantitative or qualitative impact evaluation designs)
- Use of Impact Evaluation Results (highlighting three themes crucial for effective utilization of evaluation results).
-
UNICEF. Impact Evaluation Series. Florence: UNICEF Office of Research, 2014.
ADVANCEDINTERMEDIATE
This series of briefs and videos on impact evaluation, developed by UNICEF in collaboration with RMIT University, BetterEvaluation and 3ie, covers the building blocks of impact evaluation. There are ten briefs with videos on specific topics related to Overview of Impact Evaluation, Strategies for Causal Attribution and Data Collection and Analysis Methods in Impact Evaluation.
Overview and Introduction to Types of Evaluation
-
DFID Evaluation Department. “Types of Evaluation.” In Guidance for Evaluation and Review for DFID Staff. London: DFID, 2005.
BEGINNER
An overview of the different types of evaluation available according to three broad categories: when they take place, the processes used; and where they focus.
-
Church, Cheyanne, and Mark Rogers. “Decision: Formative, Summative or Impact, What Type of Evaluation will it be?” In Designing for Results: Integrating Monitoring and Evaluation in Conflict Transformation Programs, 110. Washington, D.C.: Search for Common Ground, 2006.
BEGINNERINTERMEDIATE
Church and Rogers discuss formative, summative and impact evaluation, providing a short overview of these three types of evaluation and their relation with peacebuilding programming through some examples.
Developmental Evaluation
-
Patton, Michael Quinn. “Planning and evaluating for social change: An evening at SFU with Michael Quinn Patton.” Filmed May 2012. Simon Fraser University Continuing Studies video, 1:27.
BEGINNER
This is a lecture on developmental evaluation followed by a Q&A session where Michael Quinn Patton. This video can be useful for both evaluators and evaluation managers.
-
Patton, Michael Quinn. “Evaluation for the Way We Work.” The Non Profit Quarterly, Spring (2006).
BEGINNER
This short article explains what developmental evaluation is and how it is different from traditional evaluation.
-
Gamble, Jamie A. A. A Development Evaluation Primer. Canada: The J.W. McConnell Family Foundation, 2008.
BEGINNERINTERMEDIATE
This ‘primer’ introduces the concept of developmental evaluation and provides tools to foster its use. The first part of the book discusses the basics of DE, as well as a series of myths around it, and highlights some of the conditions needed to assess if organizations are in an appropriate space to apply DE. The second part of the book focuses on how to apply developmental evaluation, and discusses the key features of a developmental evaluator, as well as tools, issues and challenges.
Formative Evaluation
-
Patton, Michael Quinn. “Formative Evaluation.” In Evaluation Flash Cards: Embedding Evaluative Thinking in Organizational Culture, Flash Card 16. St. Paul, MN: Otto Bremer Foundation, 2014.
BEGINNER
This flash card provides the concept of formative evaluation and provides examples of formative evaluation questions.
Summative Evaluation
-
Patton, Michael Quinn. “Summative Evaluation.” In Evaluation Flash Cards: Embedding Evaluative Thinking in Organizational Culture, Flash Card 17. St. Paul, MN: Otto Bremer Foundation, 2014.
BEGINNER
This flash card provides the concept of formative evaluation and provides examples of formative evaluation questions.
What is Peacebuilding Evaluation and How has it Evolved?
The Evolution of the Peacebuilding Evaluation Field
-
Church, Cheyanne. “Peacebuilding Evaluation: From Infancy to Teenager.” New Routes 13 (2008): 3-6.
BEGINNER
This article provides an overview of the evolution of peacebuilding evaluation during the past decade, starting with a discussion on what peacebuilding evaluation is and what it focuses on. It also identifies key developments that need to occur in order to make peacebuilding evaluation more effective.
-
Church, Cheyanne. “Evaluating Peacebuilding: Not Yet All It Could Be.” In Advancing Conflict Transformation: The Berghof Handbook II, edited by Beatrix Austin, Martina Fischer, and Hans J. Giessmann, 459-482. Opladen/Farmington Hills: Barbara Budrich Publishers, 2011.
INTERMEDIATE
This article explores the state of the art in evaluation in peacebuilding, and provides a more comprehensive overview of the field, including a discussion on what makes peacebuilding evaluation different. The article also touches on the actors involved in peacebuilding evaluation; highlights some open questions and challenges; and presents a number of recommendations for advancing the quality of peacebuilding evaluation.
-
Blum, Andrew. “Improving Peacebuilding Evaluation: A Whole-of-Field Approach.” Special Report 280, United States Institute of Peace, 2011.
INTERMEDIATE
Andrew Blum highlights the evolution of the field during the past decade, but also argues that a number of problems –and the vicious circles they create– are holding back progress in the field. Finishing with a series of positive steps that need to be taken (and are being taken) by different actors in the field, the article helps to understand some of the most recent developments and challenges in the field.
What is Evaluation and Why is it Useful?
About Evaluation
-
Ed. Dr. Zint, Michaela. “Evaluation: What is it and why do it?” My Environmental Education Evaluation Resource Assistant (MEERA).
BEGINNER
This introduction to evaluation discusses whether we should or not evaluate programs, and what type of evaluation should be conducted and when. Additionally, it highlights the features of a good evaluation and discusses how to make evaluation an integral part of programming.
-
Gopal, Srikanth, and Hallie Preskill. 2014. “What is Evaluation, Really?” The FSG Blog, June 16.
BEGINNER
This article introduces the reader to the concept of evaluation, briefly discussing the question of “what evaluation is and what it is not.” More importantly, the authors address the issue of the importance of evaluating social interventions.
-
Shackman, Gene. “What is Program Evaluation? A Beginner’s Guide.” Free Resources for Program Evaluation, 2012.
BEGINNER
This guide presents a series of key questions related to program evaluation, namely: It presents an introduction to evaluation, from the concept of evaluation itself to details including what methods to use (and their advantages and disadvantages).
What Books and Toolkits Should I have on my Shelf as Resources? “Top Five” Key Books and Toolkits: Library Must Haves
Peacebuilding Evaluation
-
Paffenholz, Thania, and Luc Reychler. Aid for Peace: A Guide to Planning and Evaluation for Conflict Zones. Baden-Baden: Nomos, 2007.
INTERMEDIATE
Aid for Peace provides conceptual thought and practical support. It leads both practitioner and academic reader through a planning and evaluation process that helps the user to better design development, humanitarian, and peacebuilding interventions in conflict prone areas of the world.
[Follow link to purchase.] -
OECD-DAC. Evaluating Peacebuilding Activities in Settings of Conflict and Fragility: Improving Learning for Results, DAC Guidelines and Reference Series. Paris: OECD Publishing, 2012.
INTERMEDIATE
This guidance provides comprehensive guidance on the core steps in planning, carrying out and learning from evaluation. Additionally, it also presents a conceptual framework, discussing the need to evaluate programming in settings of conflict and fragility, and how to overcome challenges to evaluation. Annexes include a discussion on conflict analysis and its use in evaluation, a discussion on theories of change and how to evaluate them, and a sample of terms of reference of an evaluation.
For Evaluation Managers
-
Willard, Alice. “Managing and Implementing an Evaluation. Guidelines and Tools for Evaluation Managers.” Catholic Relief Services (CRS) and the American Red Cross, 2008.
INTERMEDIATE
This detailed guide provides evaluation managers with solutions on how to implement evaluations. The module focuses on what needs to be done throughout the evaluation process to manage the evaluation team and minimize the inevitable disruptions to the project’s own implementation plan.
Where Can I Find Good Comprehensive “How To” Toolkits and Resources Available Online?
Toolkits and Resources
-
Church, Cheyanne, and Mark Rogers. Designing for Results: Integrating Monitoring and Evaluation in Conflict Transformation Programs. Washington, D.C.: Search for Common Ground, 2005.
BEGINNERINTERMEDIATE
Church and Rogers’ Designing for Results is a comprehensive manual that walks the reader through the key concepts, tools and methods required to improve peacebuilding programming through better design, monitoring and evaluation. The manual is easy to read and different parts can be consulted separately as needed. The manual also highlights the importance of learning, and provides detailed explanations on different aspects of program design along with useful practical examples.
Part I linked above; Part II is available here. -
BetterEvaluation (2014). BetterEvaluation Rainbow Framework and Planning Tool.
BEGINNER
The BetterEvaluation Rainbow Framework helps to plan an evaluation by providing a clear structure that prompts the reader to think about all the steps required to perform an evaluation – from thinking about who will conduct it to analyzing results and generalizing findings.
-
Search for Common Ground (2013). Training Modules for Design, Monitoring and Evaluation for Peacebuilding.
BEGINNER
Based on Church and Rogers’ Designing for Results, these training modules synthesize key concepts from the book, complementing it with practical, real-life examples, making the content more accessible.
-
Corlazzoli, Vanessa, and Jonathan White. “Back to Basics: A Compilation of Best Practices in Design, Monitoring & Evaluation in Fragile and Conflict-affected Environments” CCVRI Guidance Series. London: DFID, 2013.
INTERMEDIATE
This comprehensive, up-to-date guide discusses the key concepts and elements of program design, monitoring and evaluation in fragile and conflict environments, and goes over the key steps needed to ensure solid Design, Monitoring & Evaluation systems. The guide offers key advice on why each of the elements discussed matter, includes practical tips, and features a detailed bibliography on relevant materials by topic.
-
Kusters, Cecile S. L., et al. Making Evaluations Matter: A Practical Guide for Evaluators. Wageningen: Centre for Development Innovation, Wageningen University and Research Centre, 2011.
INTERMEDIATE
Intended mainly for evaluators in the development field (though still relevant for other users), this document contains practical guidance on how to make evaluations more useful. It offers key principles and approaches on how evaluators can make evaluations matter more to their intended users, and how to get relevant stakeholders involved in the process.
-
Kusek, Jody Zall, and Ray C. Rist. Ten Steps to a Results-Based Monitoring and Evaluation System: A Handbook for Development Practitioners. Washington, D.C.: World Bank, 2004.
INTERMEDIATE
The primary intended users of this handbook are officials in charge of managing for results. The handbook provides a on a comprehensive ten-step model that guides the reader through the process of setting up effective results-based M&E systems. Chapter 7 focuses on key aspects of evaluation and their relevance to results-based management systems.
Video here. -
My M&E. Evaluation Step by Step.
BEGINNERINTERMEDIATE
This practical step-by-step guide offers complete guidance for evaluators and evaluation managers, including: how to plan, how to manage, how to conduct an evaluation, how to assure the quality of evaluations, and how to use evaluation results. The guide offers several technical notes on different subjects, along with suggestions of key reference manuals, useful checklists and practical examples.
-
Patton, Michal Quinn. Evaluation Flash Cards: Embedding Evaluative Thinking in Organizational Culture. St. Paul, MN: Otto Bremer Foundation, 2004.
BEGINNER
These flash cards address fundamental concepts and elements in a very clear and succinct manner. The inclusion of “bottom line” messages as well as the concrete tips and advice offered throughout the document are particularly useful.
Where can I Find Additional Resources?
Websites Dedicated to M&E or M&E of Peacebuilding
-
DME for Peace
DME for Peace was created to provide peacebuilding professionals with a platform to share tools, methodologies and findings among the community to help them identify and demonstrate what works, what does not and why.
-
Better Evaluation
This comprehensive site features a significant amount of up-to-date resources, toolkits and guidance materials on evaluation, along with blogs and discussions on the latest in evaluation where practitioners and academics share their knowledge and experiences.
-
My M and E
My M&E is an interactive Web 2.0 platform to share knowledge on country-led M&E systems worldwide. In addition to being a learning source, My M&E facilitates the strengthening of a global community, while identifying good practices and lessons learned about monitoring and evaluation in general, and country-led M&E systems in particular.
-
Genuine Evaluation
This blog by highly respected evaluators Patricia Rogers and Jane Davidson addresses a range of evaluation topics in simple, understandable form—with a focus on complex situations.
What if I don’t Understand What a Term or Word Means? Are there Glossaries of Evaluation Terminology?
Glossaries on Design, Monitoring and Evaluation
-
OECD-DAC Working Party on Aid Evaluation. Glossary of Key Terms in Evaluation and Results-Based Management. Paris: OECD, 2010.
BEGINNER
This complete glossary is intended to clarify concepts and to reduce the terminological confusion in the field of evaluation and results-based management. This document is relevant for all evaluation stakeholders. The glossary has sections in English, French, and Spanish.
-
UNFPA Division for Oversight Services. “Glossary of Planning, Monitoring and Evaluation Terms.” In The Programme Manager’s Planning Monitoring and Evaluation Toolkit, Tool 1, UNFPA Division for Oversight Services, 2004.
BEGINNER
This tool clarifies the significance and different modalities of stakeholder participation in program monitoring and evaluation
Are there Quality Standards for Evaluations?
Evaluation Standards
-
United Nations Evaluation Group (UNEG). “Standards for evaluation in the UN System.” Foundation Document, UNEG. 2005.
BEGINNERINTERMEDIATE
- These standards build upon the Norms for Evaluation for the UN system. They are drawn from the best practices of UNEG members. They are intended to guide the establishment of the institutional framework, management of the evaluation function, conduct and use of evaluations.
-
OECD-DAC Network on Development Evaluation. “Quality Standards for Development Evaluation DAC Guidelines and Reference Series.” Paris: OECD-DAC, 2012.
BEGINNERINTERMEDIATE
The OECD DAC Standards provide a guide to good practice in development evaluation. Built through international consensus, the Standards are intended to serve as an incentive and inspiration to improve evaluation practice.