“With networked research, all can help collect and share the data that is sorely lacking... We need more hands and minds to confront theory with evidence on major policy issues. This is the direction that I want the World Bank to take. This is democratizing development economics.”
- Robert Zoellick, President of the World Bank Group,
B.K. Bangash / AP
The World Bank’s Open Data initiative has demonstrated—in spades—that universally accessible data can provoke new research questions and turn conventional wisdom on its head. However, for a variety of reasons, donors seldom release comparable project evaluation data. The scarcity of reliable project-level evaluation data has created an important gap in the aid effectiveness literature. While economists and political scientists have undertaken hundreds and hundreds of econometric studies to assess the impact of aggregate aid flows on various development outcomes, the research community still knows relatively little about the project-level determinants of successful donor-sponsored projects.
In a huge step forward for aid transparency, the World Bank's Independent Evaluation Group recently published its entire store of approximately 10,000 WB project evaluations from the 1960s to present. And importantly, the Bank’s unique project identification system allows users to track individual database records back to project documents.
Shortly after the Bank released these records, we geo-coded all of the World Bank’s publicly available project evaluation data in Afghanistan since the fall of the Taliban in 2001. By “mashing-up” these geo-coded data and other statistical sources, we may begin exploring the spatial determinants of aid effectiveness in Afghanistan.
To conduct an initial "plausibility probe" of the popular hypothesis that security is a key determinant of successful projects, we overlaid all geo-coded and IEG-evaluated World Bank projects from 2002-2007 with sub-national violence data from the Long War Journal.
The resulting map reveals a puzzling pattern. The spatial distribution of violence and project performance do not correspond as closely as one might expect. Conventional wisdom holds that aid projects are generally less successful in conflict-affected areas. But this map suggests that many failed World Bank projects actually cluster in the relatively less violentprovinces north of Kabul. Additionally, this map calls attention to the fact that a fair number of World Bank projects succeed in the country's most violent southern provinces, e.g. Kandahar and Helmand.
Several explanations may account for this unusual pattern. Projects in the most dangerous provinces may receive a higher level of donor supervision (since they are located in areas where "the stakes are highest"), which previous research identifies as an important predictor of project success. It could also be the case that donor supervision is lowerin these areas, which makes it easier for local officials to avoid micro-management from Western capitals and tailor projects to local needs and conditions.
The key point is that aid effectiveness scholars cannot answer a puzzling question like this one until they know it exists. This is why we have expressed great enthusiasm for the World Bank's ambitious effort to "liberate" development data and promote "networked research". Finally, we should acknowledge that more comprehensive, time-series data from alldonorsin Afghanistan would provide a much stronger empirical basis for systematic hypothesis testing. A recent pilot project in Malawi strongly suggests that geocoding the universe of aidis feasible when donors agree to disclose detailed project documentation. However, mobilizing the necessary political will and capacity necessary to ensure that project evaluation documents are placed in the public domain will likely prove far more challenging. The latest Publish What You Fund benchmarking exercise demonstrates that only a handful of donors receive high scores on evaluation disclosure practices.