Evans School Policy Analysis & Research Group (EPAR)

This report reviews approaches to results measurement used by multilateral and bilateral donor organizations and highlights trends and gaps in how donors measure and report on their performance. Our review consists of assessing donor organizations in terms of their institutional design and levels of evaluation for results measurement, their organizational processes for measuring types of results including coordination and alignment with recipients, outputs and implementation, outcomes and impacts, and costs and effectiveness, and their processes for reporting and using results information. We collect evidence on 12 bilateral organizations and 10 multilateral organizations. The evidence review includes multi-country reviews of aid effectiveness, peer reviews by other donor organizations, donor evaluation plans and frameworks, and donor results and reporting documents. The report is based on an accompanying spreadsheet that contains the coded information from the 22 donor organizations. We find that donors report several types of results, but that there are challenges to measuring certain results at the aggregate donor level, due to challenges with funding and coordination for results measurement at the project, country, portfolio, and donor levels. Approaches to results measurement vary across donor organizations. We identify some trends and differences among groups of donors, notably between bilateral and multilateral donors, but overall there are no clear delineations in how donors approach results measurement. 

Aid results information is often not comparable, since monitoring and evaluation frameworks, information gathering processes, and definitions of “results” differ across donors and governments. This report reviews approaches to results monitoring and evaluation used by governments in developing countries, and highlights trends and gaps in national monitoring and evaluation (M&E) systems. We collect evidence on 42 separate government M&E systems in 23 developing countries, including 17 general national M&E systems and 25 sector-specific national M&E systems, with 14 focused on HIV/AIDS, 8 on health, and 3 on agriculture. The evidence review includes external case studies and evaluations of M&E systems, government M&E assessments, M&E plans, strategic plans with an M&E component, and multi-country reviews of M&E, accountability, and aid effectiveness. We evaluate harmonization of government and development partner M&E systems, coordination and institutionalization of government M&E, challenges in data collection and monitoring, and analysis and use of results information. We also report on key characteristics of M&E systems in different sectors. 

Aid donors are interested in the arguments for allocating aid via bilateral versus multilateral channels, and specifically in understanding which channel is more “effective” at supporting positive development and social outcomes. We contribute to the literature on this subject by summarizing recent OECD data on aid flows and reviewing the theoretical arguments from the aid literature on the different characteristics supporting effectiveness of bilateral versus multilateral aid. We then review the empirical literature, analyzing 40 papers that study the effectiveness of different aid channels on various outcomes. Many studies do not directly compare the effectiveness of aid channels, and the studies vary in how aid channels are defined, measured, and evaluated. Further, these studies do not directly test the hypothesized advantages of one channel of aid versus another; rather they test bilateral versus multilateral aid flows associations with development outcomes, assuming some causal mechanism is at work to explain differences in impact. We evaluate studies reporting the impacts of aid on GDP growth, governance, government investment spending, health, the HDI, poverty, and private investment, and find no consistent evidence that either bilateral or multilateral aid is more effective. The lack of conclusive evidence supporting either aid channel is likely related to differences in the methodologies of the studies included in this review, but may also be due to differences in how the theoretical arguments for the effectiveness of either channel apply in different circumstances.

This literature review examines the environmental impacts of water buffalo in pastoral and mixed farming systems in Sub-Saharan Africa, South Asia, and South America). The environmental impacts of water buffalo are less widely studied than those of the other livestock species included in this series; typically, the environmental impacts of water buffalo are incorporated into discussions of cattle without more detailed impacts being broken down by bovine type. In Asia and India, where the majority of buffalo are raised, buffalo are typically kept in small herds of only a few animals, which may minimize the local impacts of their grazing on vegetation, soil erosion and water pollution. Some aspects of buffalo feeding and life cycle patterns, as observed in the Amazon, may cause their greenhouse gas emissions to differ from those of cattle: buffalo can fatten on a wider range of grasses, reach market size in a shorter time, transition better from dry to wet seasons, and are more resistant to bovine diseases. While buffalo grazing and trampling can lead to land degradation, buffalo can contribute to nutrient and resource cycling in farming systems because their manure is considered good fertilizer and they can remove and utilize biomass grown on agricultural plots. Mitigation strategies vary by category of environmental impact, but largely suggest improved productivity to reduce land conversion, modified management systems (e.g., biodiversity, water use and consumption, farm and pastures, and waste), and the reduction of livestock numbers altogether.