Evans School Policy Analysis & Research Group (EPAR)

This report reviews approaches to results measurement used by multilateral and bilateral donor organizations and highlights trends and gaps in how donors measure and report on their performance. Our review consists of assessing donor organizations in terms of their institutional design and levels of evaluation for results measurement, their organizational processes for measuring types of results including coordination and alignment with recipients, outputs and implementation, outcomes and impacts, and costs and effectiveness, and their processes for reporting and using results information. We collect evidence on 12 bilateral organizations and 10 multilateral organizations. The evidence review includes multi-country reviews of aid effectiveness, peer reviews by other donor organizations, donor evaluation plans and frameworks, and donor results and reporting documents. The report is based on an accompanying spreadsheet that contains the coded information from the 22 donor organizations. We find that donors report several types of results, but that there are challenges to measuring certain results at the aggregate donor level, due to challenges with funding and coordination for results measurement at the project, country, portfolio, and donor levels. Approaches to results measurement vary across donor organizations. We identify some trends and differences among groups of donors, notably between bilateral and multilateral donors, but overall there are no clear delineations in how donors approach results measurement. 

Aid results information is often not comparable, since monitoring and evaluation frameworks, information gathering processes, and definitions of “results” differ across donors and governments. This report reviews approaches to results monitoring and evaluation used by governments in developing countries, and highlights trends and gaps in national monitoring and evaluation (M&E) systems. We collect evidence on 42 separate government M&E systems in 23 developing countries, including 17 general national M&E systems and 25 sector-specific national M&E systems, with 14 focused on HIV/AIDS, 8 on health, and 3 on agriculture. The evidence review includes external case studies and evaluations of M&E systems, government M&E assessments, M&E plans, strategic plans with an M&E component, and multi-country reviews of M&E, accountability, and aid effectiveness. We evaluate harmonization of government and development partner M&E systems, coordination and institutionalization of government M&E, challenges in data collection and monitoring, and analysis and use of results information. We also report on key characteristics of M&E systems in different sectors. 

This brief analyzes the indicators used by the World Bank in its Project Appraisal Documents (PAD) to measure the outputs and outcomes of 44 Water, Sanitation and Hygiene projects in Africa and Asia from 2000-2010.  This report details the methods used to collect and organize the indicators, and provides a brief analysis of the type of indicators used and their evolution over time. A searchable spreadsheet of the indicators used in this analysis accompanies this summary. We find that some patterns emerge over time, though none are very drastic. The most common group of indicators used by the World Bank are “management” oriented indicators (28% of indicators). Management indicators are disproportionately used in African projects as compared to projects in Asia. Several projects in Africa incorporate indicators relating to legal/regulatory/policy outcomes, while projects in Asia do not. In recent years, the World Bank has used fewer indicators that measure service delivery, health, and education and awareness.

In recent years, product supply chains for agricultural goods have become increasingly globalized. As a result, greater numbers of smallholder farmers in South Asia (SA) and Sub-Saharan Africa (SSA) participate in global supply chains, many of them through contract farming (CF). CF is an arrangement between a farmer and a processing or marketing firm for the production and supply of agricultural products, often at predetermined prices. This literature review finds empirical evidence that demonstrates that the economic and social benefits of CF for smallholder farmers are mixed. A number of studies suggest that CF may improve farmer productivity, reduce production risk and transaction costs, and increase farmer incomes. However, critics caution that CF may undermine farmers’ relative bargaining power and increase health, environmental, and financial risk through exposure to monopsonistic markets, weak contract environments, and unfamiliar agricultural technologies. There is consensus across the literature that CF has the best outcomes for farmers when farmers have more bargaining power to negotiate the terms of the contract. In reviewing the literature on CF, we find a number of challenges to comparing studies and evaluating outcomes across contracts. This literature review summarizes empirical findings and analyses regarding contract models and best practices to increase farmers’ bargaining power and decrease contract default.