Trócaire Somalia, Final Evaluation for the ECHO Project Consultancy

Project title: Addressing health, nutrition and protection needs of vulnerable populations in Dollow, Luuq, and Belethawa districts of Gedo region, Somalia.

Proposed assessment dates: 1st January to 28th February 2023

1. INTRODUCTION

1.1 Background

Trócaire is an Irish organisation that works in partnership with communities in over 20 countries to

• Defend human rights,
• Achieve climate and environmental justice,
• women’s girls’ protection, voice, and influence,
• Save lives and protect human dignity and
• Mobilize the public to achieve global justice

Since 1992, Trócaire Somalia has been providing comprehensive, safe, and inclusive humanitarian and development programme in the Gedo region of the Southern Somalia by integrating health, nutrition, WASH, protection, resilience, and basic education to address needs among the most marginalised and vulnerable people during emergencies and building resilience of communities to prepare for and mitigate future emergencies.

Trócaire’s work is best understood as taking a long-term developmental approach in a complex emergency, creating the conditions for recovery, by linking relief and development. In Somalia and particularly in the Gedo region, Trócaire’s continues to implement quality, sustainable and harmonised health, nutrition, WASH, education, and food security programmes to address acute livelihood and humanitarian needs.

1.2 Overview of the Project

This project aimed to improve health, nutrition and protection needs of the population in Dollow, Luuq and Belethawa districts of Gedo region. The intervention was to complement existing health, nutrition, and protection services with a view to improving existing health and nutrition gaps to reach more vulnerable and harder to reach populations in the region.

The project’s purpose therefore was to enhance access to quality health care, nutrition, and protection services to marginalised and vulnerable populations with extreme poverty index. Overall, the project intended to ensure improved integrated community case management (ICCM) model, integrated management of acute malnutrition (IMAM) and clinical response to gender based violence in remote and hard to reach zones coupled with perennial droughts and decreased livelihoods.

The key activities of the project included:

• Training health workers on Integrated Disease Surveillance and Response (IDSR).
• Immunisation coverage against measles in children under five years.
• Joint quarterly supervisions undertaken with DHBs and MOHs in all ECHO project sites.
• Recruitment and training of, ICCM supervisors
• Establishment, training and empowerment of community structures like community health workers and community health committees.
• Community-based treatment for children for malaria, diarrhoea and/or acute, respiratory infections.
• Establish and strengthen community health units
• Conduct regular community dialogue/feedback sessions and community action days
• Treating pregnant women referred by CHWs for ANC services.
• Aid SGBV victims in receiving help in less than 72 hours.
• Routine screening of children under five years and Pregnant and Lactating Women (PLWs) for acute malnutrition for better case management and referrals.
• Regular assessment and monitoring of PLWs treated for moderate acute malnutrition and follow ups.
• Treatment of children under five years admitted in stabilization centres for treatment of Severe Acute Malnutrition for better health outcomes.
• Undertake constant nutritional screening for PLWs and Infant Young Children Feeding (IYCF) sensitization sessions on infants with developmental concerns for caretakers.
• Support women who receive individual IYCF counselling.
• Identify SGBV cases received at OTP/TSFP centres and referred for support.
• Respond appropriately to GBV case management and referrals systems.
• Conduct Capacity building for health workers on clinical management of rape (CMR).
• Selection of target beneficiaries.
• Increased primary health care consultations in the selected project sites in specific regions.
• Regular training sessions for (ICCMs) and community health workers (CHWs) in the three regions of Luuq, Dollow and Belethawa districts.
• Conduct community awareness campaigns on Health, Nutrition, Sanitation and GBV response and support.
• Organise field visits at IDPs camps and conduct planned activities such as ICCMs, referral pathways, MUAC screening for children under five years, pregnant and lactating women.
• Capacity strengthening for Trocaire ECHO staff on safeguarding, accountability, and quality project delivery.

Project beneficiaries included:

• Pregnant and lactating women.
• Children under five years of age.
• Children over five years of age.
• SGBV victims
• IDPs
• Hard to reach communities

Overall Project Goal

To contribute to reduced morbidity, mortality, malnutrition rates and protection risks of vulnerable and marginalized girls, women, boys and men in drought affected and hard to reach areas in Luuq, Dollow, and Belet Hawa Districts in Gedo Region of Somalia.

Project Specific Objectives

1. Delivery of quality emergency health services to vulnerable girls, boys, women and men in three districts in Gedo, Somalia.
2. Delivery of essential lifesaving nutrition services to vulnerable women of reproductive age and under five children in three districts in Gedo, Somalia.
3. Improve access to safe, timely and lifesaving GBV services to women, girls, boys and men in the target locations.
4. Delivery of emergency assistance to vulnerable women, girls, boys and men in case of a hazardous event occurring in the project areas.
5. To reduce mortality rate of children under five years of age.
6. Strengthening health structures/systems.
7. Strengthening referral systems and pathways.
8. Improve ANC services of pregnant and lactating mothers in hard-to-reach areas.
9. Increase EPI coverage and update on immunisations activities.
10. Provision of CMR support to the GBV victims.
11. Provision of health and nutrition services to the malnourished pregnant and lactating women in out of reach zones linked with IYCF sessions.
12. Capacity building for health workers (ICCMs) supervisors and community health committees (CHWs).

1.3 The Rationale for the Evaluation

As the project ends, a final evaluation is being commissioned to assess the project progress towards achieving its objectives.

The evaluation will be required to track and ascertain progress against the outcomes of the project towards achievement of the overall goal of the project, identify lessons learnt, good practices and provide concrete recommendations for future similar interventions, to enhance adaptive programming.

1.4 Main Stakeholders of the Evaluation

The evaluation will involve participation of various stakeholders at various levels to draw lessons, gather data on impact of the interventions, identify challenges and successes and triangulate all information gathered. Stakeholders that will be involved include Trocaire staff, target beneficiaries/community members, community health workers, ICCM supervisors, MOH and local authorities of Dollow, Luuq, and Belethawa districts.

2. EVALUATION OBJECTIVES

Evaluation Objectives:

The objective of the evaluation will be to:

1. Assess the extent to which the project has achieved its purpose based on intended outputs and results.
2. Draw lessons learnt and good practices.
3. Make recommendations to inform similar future programming
4. Identify the gaps and unmet needs in the targeted districts. What could Trocaire, other donors or other stakeholders do differently to address the gaps?

The evaluation will be guided by the project’s specific objectives and commitments in the proposal document and log-frame.

Expected Results:

It was envisioned that by the end of the project period:

• There would be an increase in number of trained (ICCMs) and (CHWs) with health skills and capacity to sustain themselves and offer health services to the target populations.
• Improved Health, Nutrition and GBV beneficiaries in Dollow, Luuq and Belethawa districts through capacity building of CHWs and ICCMs supervisors on proper implementation of ICCMs activities by using ICCM protocol.
• Effective management of ECHO projects would be achieved in collaboration with the local stakeholders at all levels within the region and the (MOH) officials with the aim of sustaining its operations.
• Identifying the key challenges that the project is facing and make recommendations supporting the project staff on data quality.

3. KEY EVALUATION QUESTIONS

Specific Questions

Relevance

• Were the interventions of the project responsive to the needs of the beneficiaries?
• Did engagements with and feedback from beneficiaries shape the project design and implementation?
• Was the project in line with local needs and priorities, targeting the right geographic areas, and beneficiaries with the right interventions?

Effectiveness

• To what extent were the planned outputs and results achieved?
• Do the project outputs significantly contribute to the achievement of the outcomes?
• To what extent were activities done within the set timelines?

Efficiency

• How well were the inputs (funds, people, materials, time) used to undertake activities?
• Has the scale of benefits been consistent with the cost?
• To what extent were services and/or items delivered in a timely and quality manner?

Sustainability

• To what extent has the programme built on existing resources and capacities of communities?
• To what extent have the beneficiaries been involved in the project cycle?
• To what extent did the project work with or support linkages with local and regional authority?
• Was learning captured and acted upon during the life of the project?

Impact

• What changes has the project brought about in the lives of the beneficiaries?
• What changes has the project brought to the wider community?
• To what extent did the project outputs and outcomes contribute to the overall goal of the project?

Accountability

• Were appropriate systems of downwards accountability put in place and used by the project beneficiaries?
• Were the available and functional complaint and feedback mechanisms in place?
• Were beneficiaries aware of and utilizing the available complaint and feedback mechanisms?

Organizational Capacity

• Was there an appropriate system of management and communication in place to support the project staff?
• Were there robust MEAL and financial systems in place?

Gender equality and disability and minority inclusiveness

• To what extent has the project been gender-sensitive or transformative in design and implementation as per Trocaire Gender Equality Guidance?
• What concrete measures were taken in the project to increase gender equality and reduce gender inequalities?
• What concrete measure were taken in the project to increase disability and minor inclusiveness and reduce.

Integration of Health, Nutrition, and protection programming

• What has worked and what has not worked well in key components of integrated Health, Nutrition, and protection programming, such as in beneficiary targeting, referral? Why have these approaches worked or not worked and how could they be improved to achieve better protection outcomes for beneficiaries.

4. METHODOLOGY

The evaluation will be conducted in Gedo region, Somalia in Dollow, Belethawa and Luuq Districts. The evaluation will employ both qualitative and quantitative methodologies for example document review, in-depth survey with beneficiaries and their families, focus group discussions, key informant interviews and observation. These methods will ensure comprehensive gathering information from beneficiaries, the community, and relevant stakeholders and proper triangulation throughout the process. The data will be analysed and reported in line with the logical framework and commitments in the project proposal. Potential consultants will be expected to design a detailed methodology in their technical proposals outlining how the evaluation would best be undertaken.

5. SCOPE OF WORK

The consultant will be responsible for:

• Development of a brief inception report including a comprehensive evaluation protocol.
• Desk Review of existing program documents (including but not limited to proposal, monthly activity progress reports, needs assessments, IPTT, evaluation reports, secondary sources of information).
• Development of evaluation tools and upload to Trocaire’s CommCare platform.
• Data collection, analysis, and management.
• Development of draft and final report.
• General oversight of the assessment process.
• Presentation on the evaluation including feedback of findings, to Trocaire management and programme team.

Expected tasks and outputs will include:

1. Inception meeting with project staff, review of key relevant documents.
2. Develop and submit inception report/ protocol which will include an interpretation of the tasks and study design & methodologies, sampling procedures and detailed work plan.
3. Develop appropriate tools and instruments for gathering information and present to Trocaire team for approval.
4. Collect data form targeted beneficiaries and key stakeholders in the field.
5. General oversight and coordination of the evaluation process including logistical arrangements for data collection in collaboration with the Project Coordinator, Health and Nutrition Manager, MEAL Officer and MEAL Coordinator.
6. Process and analyse the data and develop a draft report. Revise the report and based on internal Trocaire technical feedback, submit a final report to the Trocaire Team.
7. The final report will follow report framework below:

• Executive Summary
• Background and Context
• Introduction
• Description of Methodology
• Main findings
• Conclusions
• Recommendations

1. Develop actionable recommendations context based which include specific recommendations based on evidence from the evaluation findings.
2. PowerPoint presentation on the evaluation to Trocaire management and programme team.

6. REQUIRED COMPETENCIES

1. Minimum 5 years’ experience in health, nutrition and protection programmes for humanitarian context.
2. Minimum 5 years’ experience in evaluations and/or related fields.
3. Experience in Somalia context is desirable.
4. Demonstrated experience in conducting assessments of humanitarian programmes preferably for health and nutrition programmes (experience of a similar assignment in Somalia will be an advantage).
5. Demonstrated experience in health and nutrition related learning.
6. Proven strong skills and experience in quantitative and qualitative methodology, and participatory approaches is mandatory.
7. Solid analytical skills with ability to correlate different data sets to actionable conclusions.
8. Result based planning, report writing, communication and diplomacy skills.
9. English language skills – both spoken and written (report should be written in English).
10. Ability to make clear presentations and disseminate findings to both technical and non-technical audience.

7. ETHICAL CONSIDERATIONS AND INTELLECTUAL PROPERTY

The evaluators should:

• Take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved.
• Ensure that the evaluation is accurate and reliable, is conducted in a transparent and impartial manner and contributes to organizational learning and accountability.
• Commit to adhering to Trócaire’s Safeguarding Programme Participant Policy and Code of Conduct.

All products arising from this evaluation will be owned by Trocaire. The evaluators will not be allowed without prior authorization in writing to present any of the analytical results as his or her own or to make use of the evaluation results for private publication purposes.

Tagged as: Somalia, Trócaire