How to explain the incidence of rigorous evaluation among antipoverty transfer programmes? The growth of antipoverty transfer programmes in developing countries has been a distinctive feature of development policy and practice in the last decade. Programmes providing direct transfers in cash and in-kind to households in poverty have sprung up in all developing regions, first in middle-income countries but more recently spreading to low-income countries. There is a growing knowledge base on the effectiveness of antipoverty transfer programmes based on the findings from experimental and quasi-experimental evaluations. But the intensity of programme evaluation varies across programmes, countries, and regions.

Two broad explanations are put forward to account for the relatively more intensive use of impact evaluation in antipoverty transfer programmes. One explanation emphasizes technical factors, with impact evaluations seen as examples of a shift towards evidence-based development policy. A second explanation emphasizes political factors, the need to persuade reluctant policy makers and electorates of the benefits associated with rules-based, effective, antipoverty transfers. In this context, rigorous programme evaluation findings could help overcome opposition to antipoverty transfer programmes; help protect programme agencies and their budgets from unwarranted political influence; and help build supporting coalitions capable of ensuring programme sustainability.

These two explanations generate two distinct hypotheses on the incidence of impact evaluations in antipoverty transfer programmes in developing countries. The first hypothesis suggests that the incidence of rigorous impact evaluations is positively correlated with government effectiveness and the presence of multilateral donors. Governments committed to evidence-based policy will seek to include strong monitoring and evaluation in the design of antipoverty transfers. Multilateral donors committed to aid effectiveness will push for rigorous impact evaluations in the programmes they support. The second hypothesis suggests that impact evaluations are more likely the greater the resistance to their introduction or scaling up. Opposition to antipoverty transfer programmes can come from many quarters, including from agencies involved with competing development programmes; from interests groups seeking to protect their position and influence within government; from politicians keen to use public policy as a means to strengthen electoral support through sub-national governments seeking to prevent the centralization of power. Well-designed impact evaluations can provide crucial ammunition to those advocating antipoverty transfer programmes, help to undermine the case of those opposing them, and facilitate their adoption.

The contrasting experiences of Latin America and sub-Saharan Africa with antipoverty transfer programmes and impact evaluations provide a good test of these hypotheses. In Latin America, the influence of political factors on the inclusion of rigorous impact evaluation protocols in the design of Mexico’s Progresa is well documented. This is in contrast to the paucity of impact evaluations of Bolsa Escola in Brazil, which emerged from municipal policy activism in the mid-1990s. In time, monitoring and evaluation protocols have become the norm in antipoverty transfer programmes, and a tool to improve the effectiveness of public agencies. In sub-Saharan Africa, on the other hand, the spread of antipoverty transfer programmes has been slower. The limited use of impact evaluations in sub-Saharan Africa can be explained by well-known capacity and resource constraints and governance deficits. The reluctance of local elites to embrace antipoverty transfers, however, would suggest a stronger incidence of evaluations. Resistance to antipoverty transfers by political elites and donor competition (internal and external) would be consistent with rigorous evaluation in SSA, but few programmes in the region have paid serious attention to impact evaluations. To understand this apparent disjunction, it is important to pay attention to the interaction of government and donors in the region. Competition among donors in a context of government resistance to antipoverty transfer programmes contributed to a ‘rush to implementation’. Resistance and delays in getting government approval for the implementation of pilot programmes have often forced donors to seek to implement pilot programmes before collecting baseline data or setting in place rigorous evaluation protocols. As a consequence most first-generation pilot transfer programmes in SSA lacked rigorous evaluation procedures. It is regrettable that pilot programmes lacked the very tools needed to generate knowledge on their feasibility and effectiveness. The fact that pilots were highly localized and were not embedded in government structures and domestic politics effectively lifted the incentives for rigorous evaluation which would have been predicted under conditions of agency competition and political opposition.

A second approach to assessing the two hypotheses on the incidence of impact evaluations in antipoverty transfer programmes interrogated a database of 143 flagship antipoverty transfer programmes from low and middle income countries including information on the presence of impact evaluations and their quality. The main findings are that donor engagement and scores from a government effectiveness index are positively correlated with the presence of impact evaluations. The number of flagship programmes in a country is also positively correlated with the incidence of evaluations. Both hypotheses find some support in these findings.

Our main conclusion is that the shift towards evidence-based development policy and the need to overcome resistance to antipoverty transfer programmes can contribute to explaining the incidence of impact evaluations among antipoverty transfer programmes in developing countries. Rigorous impact evaluation of antipoverty transfer programmes have an important role in supporting improvements in government effectiveness, but they also play a role in facilitating the political sustainability of antipoverty strategies.

*Armando Barrientos is Professor and Research Director at the Brooks World Poverty Institute, University of Manchester, UK.

**Juan Miguel Villa is PhD Candidate at the Brooks World Poverty Institute, University of Manchester, UK.

Barrientos, A. and J. M. Villa (2013), Evaluating antipoverty transfer programmes in Latin America and sub-Saharan Africa, WIDER Working Paper 2013/009: UNU-WIDER. Available from http://www.wider.unu.edu/publications/working-papers/2013/en_GB/wp2013-009/

Armando Barrientos’ Social Assistance in Developing Countries was published by Cambridge University Press in August 2013