Uniting Donors and Beneficiaries around Global Impact Analyses
Evaluation is Rarely a Time for Learning, Dialogue and Reform
(Fédérer donateurs et bénéficiaires autour d’analyses globales d’impact. L’évaluation est rarement une occasion d’apprentissage, de dialogue et de réforme)
01 / 1999
It has become apparent that the evaluation system does little to contribute to reforming aid working methods in order to make them more effective. The Sahel Club’s Co-operation 21 programme, which analyses 20 years of aid to the Sahel region, counted between 1,000 and 2,000 assessments for the overall aid system operating in the region. These documents tend to be lengthy and, for the most part, are read only by a very small number of people. The documents are often confidential and, in any case, have not been widely circulated. These days, they are difficult to obtain and are not sent (or rarely sent)from agency to agency.
The system is therefore a compartmentalised one between the various donors and their partners, and also within the agencies. The assessments are rarely openly discussed internally. When an assessment diagnosis gives rise to reforms that are hard to carry out, it is difficult to get the reforms passed through the internal system and then on to the donor institutions. The difficulty is compounded by the fact that the assessment departments rarely hold a strategic position within the Co-operation agencies. Since the assessments are not often discussed or results passed on, their impact is limited.
In the Sahel countries which benefit from the aid work in question, the confidentiality and lack of impact already discussed are far more pronounced. The assessment process is totally dominated by the donor, and the beneficiaries play a minor role. The beneficiaries or authorities often do not even receive copies of the assessment reports. As a more general rule, assessment is only occasionally discussed within beneficiary institutions and is even less frequently discussed between partners.
Lastly and most importantly, the evaluation is strictly limited to a narrow (institutional, spatial, temporal and thematic)field belonging to a particular aid tool. This, more often than not, prevents them from addressing the issue of impact effectively. Everyone knows that the indirect results - replicability, distribution and sustainability - are fundamental to development work.
It is often outside the scope of aid work initiatives that the most important results are to be noted. Water and soil conservation work in Burkina Faso is seen to be more successful when the spread of certain techniques (small dikes)beyond the scope of the projects is taken into consideration than through analysing whether the work done during the project is well-founded. On the other hand, the negative impact of breaking up certain ministries into small groups, resulting in the destabilisation of governmental bodies, can only be measured if the projects are viewed globally, rather than viewing each of the projects separately.
Evaluation generally gives us very little information about the long-term impact, the impact of all the aid rather than each aid project in particular, about the impact of aid work outside of its targeted field (spread of knowledge and innovation)or about the institutional, social or economic context of the projects. These aspects are a fundamental tool for discussion with partners and for orienting any possible aid reforms.
Mots-clés
Commentaire
It is not surprising that the lack of collective memory in Co-operation systems is regularly deplored. The role of learning, discussion and even measurement of the impact of evaluation is of very secondary importance in most cases. Assessment is first and foremost an internal procedure (and sometimes even a ritual)for providers of funds, whose main objectives (which are entirely valid)are to help decision making about redirecting aid work or creating new projects, and sometimes even to assess whether aid work conforms to regulations (audit). It is a precise tool that is part of the project or programme cycle, which assesses particular individuals’ work, and which is confidential, does not contribute much to reinforcing partnerships, common memory and to altering lack of interest in measuring overall impact. This results from the fact that assessment has been assigned a very narrow and specific mission.
Agencies need these assessments for administrative purposes, but they are both demanding of resources and provide little useful information. They should be replaced by other tools which are firmly oriented towards being informative for everyone, toward becoming aware of overall and long-term impact and communication between partners.
For a Sahel country, for example, the aid system - involving about 20 providers of funds, with around 100 projects and another 100 NGOs - is trying to obtain thousands of micro-results at the same time. Assessing the system by referring to the multitude of individual results runs the danger of being an endless battle for the Sahel and donor institutions, while causing the overall wider impact to be forgotten.
Instead, assessment should be seen as the opportunity for a periodic change of perspective - and no longer taking a stance based on the limited standpoint of the aid tool - with the experts, beneficiaries, field of expertise, and expected results - but rather working from the overall and long-term point of view of the beneficiary country, sector or region. Of course, an effective national assessment system would be the perfect solution. However, in most cases we are still a long way from that and we need to count on a change in donors’ assessment systems.
These types of tools are practically non-existent at present. Assessment conducted jointly between donors are extremely rare (except when the project was originally a joint one). Each assessment examines a piece of the puzzle in detail but none of these tools seek to assess whether the pieces fit together as a whole.
Collective impact assessments by theme and by sector could be used for discussion, learning and partnering. Their results could then be properly appropriated and transmitted by the participants involved so as to have a real effect on reform within the Co-operation system.
Notes
The Co-operation 21 programme is an attempt to review the 20 years of aid to the Sahel region. It was run by the Secretariat of the Club du Sahel (Sahel Club)(OECD/Paris).
[[Written for the public debate "Actors and processes of the cooperation", which could feed the next Lome Convention (European Union/ACP countries relations). This debate, animated by the FPH, has been started by the Cooperation and Development Commission of the European Parliament and is supported by the European Commission.]
Source
Texte original
DIAL (Développement Institutions et Analyses de long terme) - 4 rue d’Enghien, 75010 Paris, FRANCE - Tél. 33 (0)1 53 24 14 50 - Fax 33 (0)1 53 24 14 51 - France - www.dial.prd.fr - dial (at) dial.prd.fr
