In 2002, then-President George W. Bush announced the need for “a new compact for global development, defined by new accountability for both rich and poor nations alike.” He planned to increase U.S. foreign assistance funding to establish the Millennium Challenge Account (MCA), a new fund that was “devoted to projects in nations that govern justly, invest in their people and encourage economic freedom. Two years later, in January of 2004, Congress established the Millennium Challenge Corporation (MCC) as a bilateral foreign aid agency to administer the MCA. The MCC began putting the ideas of the MCA in action, distributing aid by competitive selection based on 17 quantitative indicators in the form of compacts, or 5-year grants. Some countries responded to the incentives procedure well; as former MCC employee, and current AidData Co-Executive Director, Brad Parks explained in this week’s ”Deeper Than Data” episode, there was “spectacular variation across all these different types of countries in how they responded to this incentive.”
While some governments responded enthusiastically to reform incentives and even incorporated them into national planning, others rejected the idea external actors, U.S. government and otherwise, could or should be a part of monitoring and evaluating policy and development. That variation led Parks and a team of researchers at William and Mary to launch a survey-based research project to measure the influence of the MCC. The team surveyed 640 MCA stakeholders in low- and middle-income countries, and found that “policymakers and practitioners in developing countries are far less allergic to the idea of performance-based aid mechanisms and modalities than development scholars.”
Measuring the Impact
The MCA stakeholder survey also proved that a large-scale survey approach was feasible to study the opinions of development policymakers and practitioners. This innovative methodology led the way for the development of the Reform Incentives project at The College of William & Mary’s Institute for the Theory and Practice of International Relations, which launched a second, larger survey to over 45,000 development practitioners and policymakers around the world in fall of 2014. Reform Incentives was later integrated into AidData’s Policy team in January of 2015, and in April of that year the team launched the Marketplace of Ideas for Policy Change report. The report analyzes the influence of over 100 external assessments according to over 6,750 survey participants, and it makes some interesting observations about factors that make assessments more and less influential.
Respondents to the Marketplace of Ideas report came from diverse backgrounds, from governments, the development sphere, and the private sector.
Assessments are more impactful in countries that are small, democratic, reform-minded, and aid-dependent, and they are more impactful when they come from external actors that are more familiar and better established. Additionally, assessments are more likely to be taken into account when they are created with data domestically sourced from the host government. Finally, people believed assessments were most influential when they actually helped a government identify and solve country-specific problems.
These are just some of the many observations made in the Marketplace of Ideas for Policy Change report; you can read a more detailed description of outcomes in the executive summary. These are only the results of the first half of the survey, though. AidData’s Policy team is planning to launch a second major report in the fall of 2015 about low- and middle-income country policymakers and practitioners’ experiences with development partners.
The Future of the Survey Practice
Beyond that, the team plans to produce biennial omnibus surveys much like this year’s that track survey respondents’ experiences with external assessments, development partners, development goals like the UN’s Sustainable Development Goals (SDGs). They also plan to produce shorter, more timely snap polls to ask tailored questions of stakeholders or policymakers specializing in more specific sectors, regions, or issues.
In order to properly establish country ownership and ensure that development activities are in line with the priorities and ideas of low- and middle-income countries, we need a new way of communicating and collaborating with policymakers and practitioners in-country. We need to make sure that development partners are considering the wants and needs of host governments. The survey-based approach employed by AidData’s Policy team offers a surprisingly simple way of doing that: as current AidData Policy Outreach Specialist Rebecca Latourell states, “the only one way to find out is to ask people.”