Can A Data Benchmarking Exercise Affect Positive Change Within Development?

Open data is certainly not sufficient to provoke positive change, but it is also not inconsequential. Open data can be catalytic when complementary efforts are in place to address several other important conditions.

October 24, 2013
Samantha Custer , Bradley C. Parks

Today, Publish What You Fund (PWYF) released its annual Aid Transparency Index and advanced the claim that transparent aid information is a “necessary condition to enable…social change”. However, earlier this week, Duncan Edwards of the Institute of Development Studies argued that the rosy prescription of “openness + ICTs = development outcomes” is fundamentally flawed.

The apparent dissonance between these two views is, in our view, reconcilable. Open data is certainly not sufficient to provoke positive change, but it is also not inconsequential. Open data can be catalytic when complementary efforts are in place to address several other important conditions: incentives are in place for donors to supply useful information; the intended beneficiaries of aid have an understanding of how the information supplied by donors is relevant to their interests; and local stakeholders in the developing world have the ability to put such data to effective use.

PWYF’s annual benchmarking exercise is a powerful advocacy tool to convince donor organizations to make development finance data more transparent. Minimum transparency standards, such as those established by the International Aid Transparency Initiative (IATI), can influence the information that donor agencies choose to disclose. However, in the absence of a scorecard exercise that provides a reputational reward to donors who make aid information more accessible and actionable, a “race to the top” is significantly less likely. The breadth of the 39 PWYF Index indicators – including measures at the organization, country and activity/project levels – reinforces the expectation that transparent aid must be "timely, comprehensive, and accessible".  Donors clearly pay attention to their PWYF ranking. The counterfactual is clear: without this annual benchmarking exercise, donors would likely feel less pressure to compete with their peer organizations by setting the bar for data disclosure higher each year.  

At the same time, we believe the PWYF Index would be even more compelling if it refined its approach to assessing what makes aid information useful to local government and civil society stakeholdersWhile the 2013 Index emphasizes that “more [data] is not enough”, usefulness is still frequently reduced down to accessibility (e.g., ensuring data is not in a format that is difficult to access, analyze and reuse). In fact, the relevance of development finance data to prospective end users is an equally important component of what makes information useful. In our own work, we have foundthat pinpointing and visualizing the precise sub-national locations of aid projects can help local stakeholders in developing countries more easily zero in on the activities and financial flows of greatest interest to them.

Since 2012, PWYF has included an explicit indicator in the Index assessing whether a donor has published the subnational geo-location for project/activity level documentation.  This is an important step in the right direction, but it does not provide a strong incentive for donors to make their project- and activity-level data accessible to, and actionable for, local stakeholders in developing countries. The existing indicator only captures whether a donor reports any subnational information, rather than specifying a particular level of spatial resolution (e.g. down to the village level). The few donors who do provide subnationally geocoded development finance information do so at widely varying degrees of granularity. This variation matters. While reporting at a province level is better than pure country aggregates, more localized information at a district, village or “street corner” level is substantially more relevant (and useful) to a civil society group or district government seeking to track aid flows relative to need and identify gaps in service delivery.

The inclusion of a more fine-grained indicator in next year’s PWYF Index to encourage donors to publish location information at a higher level of spatial resolution would be a move towards incentivizing the disclosure of more relevant, and hence more useful, development data. In addition, more could be done to assess donors on the degree to which they are building the awareness and capacity of governments, civil society and citizens to actually use aid information in advocacy, research and decision-making.

Samantha Custer is Director of Policy Analysis at AidData.

Brad Parks is the Executive Director of AidData at William & Mary. He leads a team of over 30 program evaluators, policy analysts, and media and communication professionals who work with governments and international organizations to improve the ways in which overseas investments are targeted, monitored, and evaluated. He is also a Research Professor at William & Mary’s Global Research Institute.

Brad Parks is AidData’s Co-Executive Director and Samantha Custer is AidData's Director of Communications & Policy Outreach.