Tutorial Video

Need More Information on Impact Evaluation Methods?

The Impact Evaluation in Practice guide is a companion piece of the Impact Evaluation Toolkit. It helps policymakers and practitioners understand available impact evaluation methods, and make decisions based on evidence of what works.

Experience from the Field: Country Spotlights (PDFs)

Our Country Spotlights give you real-life experience from the field in designing and implementing impact evaluations. Scroll over the world map and learn from what other countries have been doing.

Afghanistan: Data Access Policy [PDF, 222KB]
Argentina: Using Administrative Data [PDF, 237KB]
Benin: Design & Implementation Challenges [PDF, 285KB]
Cameroon: Sampling Frame & Political Buy-In [PDF, 392KB]
Central African Republic: Integrating Impact Evaluation & Intervention [PDF, 302KB]
Democratic Republic of Congo: Team Building & Post-conflict Settings [PDF, 291KB]
Nigeria: Why do Impact Evaluation? [PDF, 447KB]
Rwanda: 1-Using Power Calculations in Policy Dialogue/2-Team Building & Local Capacity/3-Using Results Dissemination for Policy Dialogue [PDF, 383KB]
Zambia: Data Collection Challenges [PDF, 310KB]

In this module, you will learn why it's important to evaluate Results-Based Financing interventions. You will find out how to establish a theory of change and a results chain, and how these help develop hypotheses that an impact evaluation can test through evaluation questions. You will gather food for thought by browsing through first-generation questions on whether Results-Based Financing works, and second-generation questions on how Results-Based Financing can work better. At the end of this module, you should know how to develop impact evaluation questions relevant to the context you work in. These questions should respond to key aspects of interventions and fill knowledge gaps on your topic of interest.

Module 1: Choosing Evaluation Questions - Guidelines

Toolbox

1.01 Theory of Change Graph [PDF, 46KB]
1.02 Results Chain Template [PDF, 149KB]

In this module, we will teach you how to design your impact evaluation. Steps include:

1. defining output and outcome indicators of interest that translate research questions into observable variables;
2. determining an identification strategy, including the "arms" of a study that respond to research questions and guarantee a valid comparison group;
3. making sure the sample has sufficient power to detect changes across a representative population;
4. knowing which data will be needed to measure indicators of interest;
5. how the timeline, team composition and budget must align and adjust simultaneously; and
6. how to plan for dissemination activities. At the end of this module, you will know how to design an impact evaluation.

Module 3: Designing the Impact Evaluation - Guidelines

The Toolbox

3.01a RBF Indicators 1 [PDF, 823KB]
3.01b RBF Indicators 2 [Excel, 54KB]
3.02 WHO Indicators [Zip, 2.15MB]
3.03 IE Design Paper Template [Word, 30KB]
3.04 IE Budget Template [Excel, 14px]
3.05 Ex-ante Power Calculation [PDF, 392KB]
3.06 Power Calculations Binary var. [PDF, 170KB]
3.07 Power Calculation References [Zip, 2.55MB]

In this module, you will learn essential steps that lay the groundwork for data collection. Steps include:

1. scheduling data collection,
2. reaching formal agreements ex-ante with stakeholders on data ownership,
3. making sure participants in the survey are protected by defining a research protocol and getting the study ethics right,
4. determining a data entry strategy and what's needed from survey firm(s),
5. hiring one or more survey firm(s), and, finally,
6. adapting and preparing data collection instruments and entry tools.

At the end of this module, you will be ready to go into the field.

Module 4: Preparing the Data Collection - Guidelines

The Toolbox

4.01 Gantt Chart [Excel, 14KB]
4.02 MOU on Data Access [Word, 38KB]
4.03 Research Protocol [Word, 67KB]
4.04 Informed Consent [Word, 19KB]
4.05 Health Facility Survey Firm TOR [Word, 34KB]
4.06 Household Survey Firm TOR [Word, 36KB]
4.07 Data Collection Budget [Excel, 18KB]
4.08 Equipment for Biomarker Data [Word, 15KB]
4.09 Health Facility Questionnaires [Zip, 3.64MB]
4.10 Household Questionnaires [Zip, 3.07MB]
4.11 Community Questionnaires [Zip, 97KB]
4.12 Costing Questionnaires [Zip, 265KB]
4.13 Data Entry Program [Zip, 15.8MB]
4.14 Anemia Referral Guidelines [Word, 14KB]
4.15 Anemia Referral Form [Word, 15KB]
4.16 Translate Questionnaires [Zip, 918KB]
4.17 Institutional Review Board TOR [Word, 31KB]
4.18 Accurate Translation Certification [Word, 15KB]

In this module, you will dig into data collection activities and data quality safeguards. You will see how to define the sample and sampling frame in the field and create unique identifiers for geographical areas and study arms. You will understand the benefits of data collection pre-test and proper data entry management plans. You will learn to plan for fieldwork supervision, recruit and train field teams, conduct a pilot test and, finally, how to manage fieldwork and ensure regular reporting on fieldwork activities. At the end of this module, you should know how to provide sufficient training, support, supervision and communication channels to ensure quality fieldwork activities and data.

Module 5: Implementing the Data Collection - Guidelines

The Toolbox

5.01 Interview Duration Sheet [Excel, 11KB]
5.02 Enumerator Evaluation [Excel, 16KB]
5.03 Survey Progress Report I [Word, 22KB]
5.04 Survey Progress Report II [Excel, 18KB]
5.05a Household Field Manual [Word, 2.37MB]
5.05b Household Training Program [Excel, 15KB]
5.05c Household Training [Zip, 5.79MB]
5.06a Health Facility Field Manual [Word, 206KB]
5.06b Health Facility Training [Zip, 12.7MB]
5.07 Training, CAR & Cameroon [Zip, 3.26MB]
5.08 Health Facility Supervisor Checklist [Word, 22KB]
5.09 Health Facility Arrival Checklist [Word, 20KB]
5.10 Health Facility Supervisor Form [Word, 22KB]
5.11a Daily Listing U5 Exit Interviews [Word, 21KB]
5.11b Daily Listing ANC Interviews [Word, 19KB]
5.12 Cash Management Sheet [Excel, 35KB]

In this module, you will find out why properly storing and documenting data matters, how this can be done and who is responsible for it. You will discover the benefits of the World Bank data catalog and its key features. You will also be able to distinguish which types of data should be stored, where to store it and who should access it. At the end of this module, you will know how to ensure that survey respondents are not put at risk, how to keep the outcome of the survey safe with minimal risk of damage or loss, and how to provide easy access for the impact evaluation research team.

Module 6: Storing and Accessing Data - Guidelines

The Toolbox

6.01 Data Catalog Deposit Form [Word, 126KB]
6.02 Nesstar Data Storage Templates [Zip, 28KB]
6.03 Login Data Management Toolkit [MHT, 92KB]
6.04 How to Access Data Catalog [PDF, 194KB]

In this module, you will learn the functions of each impact evaluation report (baseline, midline, endline), get useful hints on how to clean the data, create variables including indicators of interest, and how to validate the design of the impact evaluation at baseline. You will receive guidance on impact analysis to supplement the impact analysis methods presented in Impact Evaluation in Practice (Gertler et al. 2016). Finally, you will learn how to build your analysis in the spirit of efficient dissemination to key stakeholders. At the end of this module, you should be ready to start analyzing the data while both preserving its integrity and planning for sharing results.

Module 7: Analyzing Data and Disseminating Results - Guidelines

The Toolbox

7.01 Household Baseline Report
7.01a Handbook for Report [PDF, 650KB]
7.01b Indicators: Rwanda [Excel, 60KB]
7.01c Stata Do Files: Rwanda [Zip, 135KB]
7.01d Power Calculations: Rwanda [Zip, 759KB]
7.02 Health Facility Baseline Report
7.02a Suggested Report Outline [Word, 15KB]
7.03 Community Health Worker Baseline Report
7.03a Stata Do Files: Rwanda [Zip, 23KB]
7.04 Stata ado file Balance tables [Zip, 207KB]
7.05 WHO Anthro package [Zip, 748KB]
7.06 Stata training [Zip, 2.34MB]
7.07 Stata training Validate Design [PDF, 1.41MB]

In this module, you will learn how monitoring and documenting program implementation can help inform the relevance of impact evaluation design and analysis. You will discover examples of program implementation adjustments that can affect the impact evaluation, and why it is important to monitor program implementation. You will also learn how complementary data sources can help save monitoring costs and time. At the end of this module, you will know what to monitor, what can be used to monitor interventions, and how documenting results can help ensure the impact evaluation stays relevant over time.

Module 8: Monitoring and Documenting RBF Programs - Guidelines

The Toolbox

8.01 Monitoring Indicators: Rwanda [Excel, 10KB]
8.02 Field Supervision Visit: Rwanda [Excel, 24KB]