Program Area: GIE

Geospatial Impact Evaluations

Overview

Overview

Rigorous impact evaluations in a fraction of the time of an RCT

Geospatial Impact Evaluations (GIEs) measure intended and unintended impacts of development programs by leveraging readily available data like satellite observations to mimic the conditions on the ground in a randomized control trial (RCT) in order to establish a reliable counterfactual for meaningful comparisons — at a fraction of the time and cost of a “traditional” RCT.

LikeRCTs, GIEs can estimate the net effect of a specific program by comparing similar groups whose only difference was exposure to the program (or treatment).  Unlike RCTs, GIEs use precise geographic data to establish this counterfactual retroactively, eliminating the need to assign program participants into randomized treatment and control groups within the program design.

Case Studies

Roads and Transportation

AidData and USAID/West Bank and Gaza have collaborated to conduct an evaluation of the Infrastructure Needs Program (INP) II, which funded the construction of water infrastructure, road networks, and schools with the ultimate goal of improving the Palestinian economy.

Natural Resource Concessions

AidData constructed a new dataset that measures the precise locations of 557 natural resource concessions granted to investors in Liberia, and then then merge these data with a remotely sensed measure of nighttime light growth at the 1km x 1km grid cell level to analyze effects across sectors and investor types.

Indigenous Land Rights

We studied whether formalization of indigenous communities' land rights affects the rate of deforestation in both the short and medium terms as the result of KfW-funded land tenure interventions across the Brazilian Amazon from 1997 to 2008.

China and Sensitive Forests

What are the conservation impacts of Chinese development activities in ecological hotspots? We generate and sub-nationally geo-reference a dataset of official Chinese development activities implemented between 2000 and 2014 in the Tropical Andes, the Great Lakes region of Africa, and the Mekong Delta. We then merge these project data with a long series of high-resolution satellite data in order to evaluate their impacts on forest cover.

More Case Studies >

Benefits of GIE Methodology

GIEs can be implemented remotely and retrospectively, leveraging existing location-referenced data and generate portfolio-wide insights.

Features

Measure real impact

Geospatial Impact Evaluations measure intended and unintended impacts of development programs. Leveraging readily available data like satellite observations or household surveys, GIE methods establish a reliable counterfactual to measure impact - at a fraction of the time and cost of a "traditional" randomized control trial (RCT).

Methodological rigor

Like RCTs, GIEs can estimate the net effect of a specific program by comparing similar units where the only difference was an intervention, or treatment. Unlike RCTs, GIEs use precise geographic data to establish this counterfactual retroactively, eliminating the need to assign program participants into randomized treatment and control groups within the program design.

Learn more efficiently

GIEs can be completed in a fraction of the time and financial cost of an RCT by eliminating the need for customized data collection in treatment and control groups before, during and after the program.

See portfolio-wide insights

GIE methods are also flexible tools that can either be used to evaluate individual projects or project portfolios.  Whereas RCTs are often implemented in narrowly bounded settings, GIEs can be used with data for an entire country (or even multiple countries), which makes it possible to draw conclusions about impacts and cost effectiveness that are broadly generalizable.

Ascertain long-term impact, even in inaccessible places

Additionally, GIEs can be implemented remotely, retrospectively, and affordably, opening up new opportunities to measure long-run programmatic impacts, which is especially useful to evaluators working in conflict and fragile state settings.

Learn more in our #CGDTalks Video

Geospatial impact evaluations (GIEs) rely on subnationally geo-referenced intervention, outcome, and covariate data and quasi-experimental methods of causal inference to measure the intended (or unintended) impacts of development programs.

GIEs seek to mimic the conditions of a randomized control trial (RCT) with observational data. RCTs are powerful because they create conditions under which one can reliably ascertain that individuals’ participation in a program was not correlated with their outcomes. GIEs create similar conditions, but without randomly assigning individuals to treatment and control groups.  

For our in-depth exploration of GIEs, please see A Primer on Geospatial Impact Evaluations Methods, Tools, and Impact Evaluations.

Using counterfactuals to find impact

The key is making comparisons across individuals who are sufficiently similar to one another and experiencing changes that are otherwise similar.  The best way to make such comparisons is to identify comparison individuals who are geographically close to the program participants, but unlikely to be affected by the program’s presence.  Doing so requires geographically precise data on programmatic interventions and their intended (or unintended) outcomes. Such data are rapidly expanding in number, scope, periodicity, and availability.

Rather than using randomization to identify counterfactual cases, GIEs seek to achieve a similar result through one of three methods:

  1. Strategic subsampling of observational data to identify treatment and control cases that are nearly identical but for the presence or absence of the intervention (e.g. propensity score matching)
  2. Comparing the pre- and post-intervention change in the outcome of interest for a treatment group relative to a control group (e.g. difference-in-differences, fixed effects)
  3. Exploiting the discontinuity around a geographic cutoff that is “as-if random” (where the treated cases and control cases on either side of the cutoff are extremely similar across pretreatment covariates)

Featured Publications

Impact Evaluation

Evaluation of the Infrastructure Needs Program II

Ariel BenYishay, Rachel Trichler, Dan Runfola, Seth Goodman

2018-10-08

Impact Evaluation

Evaluation of the Local Governance and Infrastructure Program

Pablo Beramendi, Soomin Oh, Erik Wibbels

2018-10-08

53
Working Paper

The Donor Footprint and Gender Gaps

Maria Perrotta Berlin, Evelina Bonnier, Anders Olofsgård

2018-06-25

50
Working Paper

Chinese Development Assistance and Household Welfare in Sub-Saharan Africa

Bruno Martorano, Laura Metzger, Marco Sanfilippo

2018-05-23

Journal Article
World Development

Natural resource sector FDI, government policy, and economic growth: Quasi-experimental evidence from Liberia

Jonas B. Bunte, Harsh Desai, Kanio Gbala, Bradley C. Parks, Daniel Miller Runfola

2018-03-20

Journal Article
Joint European Conference on Machine Learning and Knowledge Discovery in Databases

Quantifying Heterogeneous Causal Treatment Effects in World Bank Development Finance Projects

Jianing Zhao, Daniel M. Runfola, Peter Kemper

2017-12-30

Featured Blog Posts

Program Team

For technical or research inquires, contact:

Research & Evaluation

Ariel BenYishay

Chief Economist, Director of Research and Evaluation

Research & Evaluation

Dan Runfola

Senior Geospatial Scientist

Research & Evaluation

Christian Baehr

Junior Data Analyst

Research & Evaluation

Miranda Lv

GIS Analyst

Research & Evaluation

Rachel Trichler

Senior Research Analyst

Research & Evaluation

Seth Goodman

Data Engineer

Partners

For partnerships and media queries, contact:

Alex Wooley

Director of Partnerships and Communications