Terms of Reference Evaluation of Concern Ethiopia's National CMAM programme (4 Regions) including the Bale (5 Woreda) emergency project implemented for the period 2007-10. 1. Background The recurring history of food crises in Ethiopia has led to serious famines in the past. The food security status of the agricultural, pastoral and agro-pastoral population in the country is constantly affected by drought. Crop and livestock production are the major sources of food in Ethiopia. In the absence of the rains, shortages of pasture and water considerably diminish the production capacity of the crops and livestock, creating food shortages and high levels of vulnerability to malnutrition. Outdated farming practices, land degradation, limited access to safe water sources all ensure that even in 'normal' food production years, many children suffer from severe acute malnutrition (SAM). The lack of adequate health services and poor caring practices of community members fuel the chronic and very high levels of malnutrition experienced in Ethiopia. The poverty reduction strategy that the Ethiopian government is currently following, is believed to be contributing to the prevention of malnutrition, with the aim of bringing sustainable development for the country. Until the poverty reduction and health development strategies realize their goals, Concern, in partnership with all levels of the MoH has been trying to reduce the consequences of SAM; through the implementation of Community Management of Acute Malnutrition (CMAM, formerly known as Community-based Therapeutic Care, CTC) programmes, at national level since July 2005. The aim of the N-CMAM programme has been institutionalization of the CMAM approach (particularly the Outpatient Therapeutic Programme (OTP)) within the regular health services of the MoH. So far, there have been varying degrees of success in integration, with issues affecting levels of integration including; the area in which the programme was implemented, how much commitment was demonstrated from key partners/stakeholders, how much input was required from Concern to achieve the expected results, the severity of emergency situations. A great deal of institutional learning has been gained, by working through the various barriers to integration that were identified in differing contexts of the 4 main regions. 2. Project Summary The National CMAM programme is seen as a longer-term strategy to build up capacity, strengthen the MoH system and move towards sustainability of the programme. In addition, as stated in the National Nutrition Strategy (implemented through the National Nutrition Programme (NNP), the approach the N-CMAM programme uses is very much in alignment with, and can comfortably support, the Ethiopian FMoH vision for increased coverage of health services. The MoH focus of the increased coverage is at community level; including decentralisation of services to community level, utilisation and involvement of Health Extension Workers, other community health agents and kebele (village) level actors, in the planning and management of services. Since December 2005, the N-CMAM programme has provided technical assistance to the MoH to implement services for SAM via a package of 'minimal support' in a steadily increasing number of woredas in the four main Regions of Ethiopia (as of July 2010, total of 99 woredas in SNNPR, Tigray, Oromiya and Amhara). The objectives of the programme, since 2005, are: a) To provide CMAM services ('minimal support') in multiple regions of Ethiopia to address severe acute malnutrition in children under 5; b) To strengthen the capacity of the Ministry of Health at regional, zonal and woreda level to provide a continuous SAM service provision, especially with regard to training, monitoring and coordination of services. The approach of 'minimal support' has consisted of theoretical and on-the-job trainings including; the principles of both in-patient and outpatient management of severe acute malnutrition; reporting and supervision, workshops and experience exchange visits and support for the scale-up of emergency nutrition activities. In addition, considerable advocacy has been conducted at both Federal and Regional level in order to facilitate the institutionalisation process, encouraging the Federal MoH to announce full 'roll out' of the Therapeutic Feeding Programme (TFP) in 2009, using the CMAM approach. The National CMAM programme also provided considerable support for Concern's emergency nutrition interventions from 2008 onwards, in terms of personnel, funding for supplementary food, training and technical support. Additionally, Concern recently piloted the scale-up of support to 'maximal' level for Woreda MoHs, to implement high-quality OTP services at time of emergency; to enable them to provide an effective response for the increased case loads that present due to rising levels of food insecurity. The 'maximal support' approach provides additional manpower, training, assistance with supplies, transport, logistics and reporting, while maintaining the Zonal and Woreda MoH as the 'owners' of the scaled-up response. In line with the CMAM approach, focus is placed on supporting the decentralization of OTP to health post (village) level, to increase coverage and access of services while maintaining service quality. The pilot was conducted in 5 Woredas of Bale Zone during September 2009-February 2010, following a request from Oromiya Regional Health Bureau to respond to the deteriorating nutrition situation in the area. 3. Overall objectives The overall objective of this assignment is to undertake a final evaluation of the National Community Management of Acute Malnutrition project to assess the effectiveness, appropriateness, impact of the intervention and whether the project has achieved its stated objective of supporting vulnerable target populations access necessary services and improve their capacity to survive a variety of shocks they are faced with. The evaluation will also make recommendations for future intervention and document challenges as well as lessons to be learnt. The evaluation will look at the N-CMAM project status beginning from 2007 up to the time of the evaluation and at the emergency nutrition support from September 2008 to February 2009; the details of the CMAM projects in terms of their processes and achievements, the extent of project integration with the Ethiopian national health structure and coordination with other partners. 4. Specific objectives - The key objectives of this evaluation are to assess: 4.1 The relevance of the programme: - Did we choose the right thing to do in the context of Ethiopia? - Is the programme in line with the needs and priorities of the most vulnerable and poorest targeted? Were the targeting criteria followed? - Is the programme consistent with the policy and strategic direction of the government of Ethiopia? - Are the activities and the outputs as reflected in the Programme Proposal consistent with the overall goal and objectives? - Has the N-CMAM programme facilitated a more timely and efficient scale up for emergency response when nutrition emergencies have arisen in its target Woredas (i.e. has the project contributed to disaster risk reduction in a disaster-prone context)? - Is the overall goal and objective of the programme as reflected in the Programme Proposal still valid? . -Were the assumptions in the Programme Proposal reasonable and appropriate? 4.2 The effectiveness of the programme: - To what extent were the objectives of the programme as reflected in the Programme Proposal achieved? - Were the activities sufficient to achieve the objectives / outputs? - What are the major factors influencing the achievement or non achievement of the objectives? - Are the assumptions as reflected in the Programme Proposal still valid? 4.3 The efficiency of the programme: - Was the programme cost effective? - Was the coordination between the Concern Ethiopia team and other international NGOs, the UN system and government organisations effective and was the engagement with these desirable/possible. 4.4 The impact/ sustainability of the programme: - Did the programme achieve what it set out to achieve? - What positive changes are observed in the lives of the target group as a result of the implementation of the programme? - Did the response reduce future vulnerabilities, particularly to acute malnutrition? - Are there factors that impede the achievement of the overall programme goal? - What are the unintended positive and negative impacts of the implementation of the programme? What measures have been and can be taken to eliminate or reduce the negative impacts? 4.5 The adherence to external standards for programme quality particularly national and Sphere standards for coverage and performance of therapeutic feeding/CMAM programmes, the Code of Conduct of the Red Cross and Red Crescent Movement and NGOs in Disaster Response and People In Need. 4.6 The success of the organization/programme in mainstreaming HIV and AIDS, Gender and Equality Did the intervention identify specific vulnerable groups like women, elderly, disabled and other identifiable vulnerabilities? What special efforts were taken to address the needs of vulnerable groups and expanding benefits to them? Was HIV and AIDS considered or addressed at any stages in the responses? 4.7 The adherence to the Programme Participant Protection Policy with specific focus on activities involving children 4.8 The monitoring and evaluation system and the appropriateness of the indicators used for this purpose. The programme currently reports against the indicators identified in the USAID/ OFDA Guidelines for Unsolicited Proposals and Reporting. 4.9 With reference to accountability to beneficiaries: - Where appropriate mechanisms developed at the local level to enable affected communities to actively participate in the design, planning, implementation and monitoring of the project? 4.10 The extent to which 'lessons' or recommendations from reviews/ field visits/ regional annual workshop of the project were incorporated into this response. 5. Specific issues to be evaluated 5.1 Assess the degree to which management and monitoring systems for CMAM have been efficiently integrated into existing health and community systems and their potential to be sustained by the regional, zonal and woreda health teams and community networks - identifying any barriers/ successes 5.2 Assess the intervention's coverage within the Woredas where coverage surveys have been conducted (CSAS and SQUEAC methodology)/ areas in terms of the proportion of SAM cases treated and number of health facilities providing services, against proposed targets and international standards. 5.3 Assess the degree to which the NCMAM's 'maximal support' [e.g. Dole Mena] was effective vs. the 'minimal support' package offered elsewhere 6. Methodology A consulting firm or a team of consultants that consists of one national and one international consultant is recommended to conduct the evaluation. The consultants are expected to outline in detail their methods and tools. Secondary information: . Review programme records and data (proposals, reports, studies, databases on admin/ discharge criteria, minutes of meetings with government agencies, donor reports, Concern's and UNICEF's supervision check lists/ scoring and pre-and post tests, coverage surveys, programme guidelines, etc) kept at Concern Head Office . Review records at sampled health centres/ health posts, to assess the programme objectives, activities and general effectiveness in the programmes differing contexts across the two Regions. . Review the training materials, training strategy, results and workshop presentations . Review the programme agreements with MoH Primary data collection: . Collect additional data / observe programme delivery at selected sites, where appropriate . Conduct key informant interviews with N-CMAM management staff, Concern Ethiopia management staff, Federal MoH, UNICEF, Zonal MoH, Woreda MoH and health centre heads of sampled target Regions; . Conduct focus group discussions and structure/ semi-structured interviews with Health Extension Workers, village community health volunteers/promoters, beneficiary groups (those enrolled in OTP) and other community members; 7. Expected outputs . De-brief on the findings with Concern and Partners before submission of the draft report . Comprehensive draft evaluation report (in both hard and electronic format on CD) written in English of maximum 30 pages excluding annexes that will contain the following: o A stand-alone executive summary of maximum 3 pages o Analysis of main findings o Targeted recommendations o Lessons to be learnt, best practices and challenges faced . Final evaluation report based on Concern's feedback- same criteria as draft report applying 8. Reporting In all matters in relation to the evaluation, the consultant will report to the National CMAM Programme Coordinator in Addis Ababa. 9. Proposed timeframe The evaluation is planned to be undertaken beginning from 15 October 2010 and is expected to take four weeks until 15 November 2010. The tentative itinerary may include (though will be decided with selected consultants) the followings: Week one: familiarize with key documents; meet Concern team, N-CMAM staff and key partner agencies. Week two: travel to Bale Zone end week one and conduct assessment on emergency programme. Assess N-CMAM in Oromia Region. Week three: travel to Tigray Region and assess the N-CMAM programme there (go to Mekele and selected Woredas (both well and poorly performing) and visit one or two health facility in each Woreda. Week four: further meetings in Addis with key partners, compilation of report, debriefing. The consultants are expected to submit the draft findings/report during the final week of the study, with the final report submitted to Concern Ethiopia within two weeks of the end of the evaluation. 10. Evaluation team profile The evaluation team includes an international consultant who should hold a second degree in public health or nutrition and a national co-evaluator. Suitable candidates should have proven experience in carrying out programmatic evaluations of a similar nature. Interested applicants: please send CV and cover email to Kate Golden (kate.golden@concern.net) and Marie Rongeard (Marie.rongeard@concern.net) by September 12, 2010