Emergency Food Security and Livelihoods Intervention Evaluation
Background
Oxfam has been providing emergency food security services to an estimated 30,000 internally displaced persons (IDPs) in the Protection of Civilians Camps (POC), in UN House, Juba South Sudan from May 2014 to December 2014. As part of emergency food security activities, Oxfam distributed vouchers, fuel efficient stoves, and start up kits for income generating activities in PoCs 1, 2 and 3 and handgrinders in POC1 and POC2. Cash vouchers were distributed per household while the quantity of milling and charcoal vouchers distributed depended on household size, complimentary to General Food Distributions (GFD). Distribution of milling vouchers was done under WFP Field Level Agreements (FLA) which was extended in October 2014. Oxfam has been implementing its charcoal voucher programme since February 2014. This was funded under two ECHO grants. The ECHO grants together with the NRC grant also funded commodity vouchers, income generating activities and fuel efficient stove production and distribution. Handgrinders were funded by Oxfam Novib. The main purpose of this evaluation is to assess whether set project objectives for food security and livelihoods interventions were met in an efficient manner, it will also look at ways of working and lessons learnt during implementation of the project
Evaluation Objectives
To assess the efficiency, effectiveness, accountability and impact of the food security and livelihoods interventions in UN House Juba in relation to set objectives (refer to Appendix 1 Log-frame)
To understand what worked well and what did not work well, gather lessons learnt and areas for improvement
To identify the potential and conditions for replication of similar interventions in other areas of South Sudan and what scale would the interventions be suitable for replication.
Methodology and Key Questions
Desk review of Echo, NRC and WFP proposals and reports, market assessments, baseline reports and post distribution monitoring reports.
Household Interviews, Focus group discussions, key informant interviews with stakeholders (beneficiaries, non-beneficiaries, vendors and other groups) and 1 day lessons learnt workshop
Below are some areas to be addressed by the evaluation that cut across all activities; milling vouchers, charcoal vouchers, cash vouchers, IGAs, handgrinders and fuel efficient stove production and distributions where appropriate:
2.1 Effectiveness and Relevance of the Intervention
Did the transfer modality and activities implemented address the beneficiaries’ needs and circumstances in terms of the objectives as stated in the project document? (Refer to Appendix 1 Log-frame). How were the differing needs of men and women taken into account?
Type of interventions implemented – was this the most cost effective way of implementing the project?
In relation to milling, was the coordination, roles and responsibilities of WFP and Oxfam well defined and appropriate?
Analyze the effectiveness of the coordination of stakeholders in the planning, implementation and monitoring components of the program.
In the opinion of beneficiaries, were their protection needs met in-terms of reduced risks associated with movement in and out of the PoC for milling/charcoal services?
Were the millers, charcoal vendors and traders accessible in terms of location and number per PoC to meet the needs of the beneficiaries?
Was the voucher amount adequate for all vouchers? And what was the redemption rate and why?
Was beneficiary targeting done in and effective manner that promoted community ownership?
In the opinion of the beneficiaries did they receive good quality milling, charcoal and trader services?
In the opinion of the beneficiaries were there other interventions that they believe could have addressed their food security needs better?
Was there a significant impact on HH income and shift in expenditure patterns i.e. spending on food, milling and charcoal before and after the interventions?
Did the beneficiaries use the vouchers as intended? If not why, what did they do with vouchers not redeemed?
Assess the quality of the distribution process (including security/protection issues for the beneficiaries.
Were issues of gender addressed at all stages of the project cycle?
Was the timing of distributions (where they are not aligned with GFD) appropriate?
What changes took place in the implementation of the programme which differed from the design.
How did the security context affect programme implementation and human resources?
2.2 Efficiency
What worked well i.e. staffing structure/ organogram, staffing needs, resources, logistics, HR, finance, funding, and management support? What did not go well and why?
Were the resources for running all the activities available, adequate and was this the best use of resources?
Were procurement processes efficient and did they take into account security and other contextual issues?
Were the interventions complimentary to each other and consequently did this enhance their efficiency?
2.3Lessons Learned
What are the lessons learnt and recommendations for improvement?
What is the perception of the intervention by General Food Distribution (GFD) partners i.e. Concern and World Vision, community leaders, camp management i.e. ACTED, IRC, Security and donors (ECHO, WFP, NRC)?
What were the links to other donor funded projects strategies, NGO plans and exit strategy internal and external.
What were the internal/external factors that affected implementation and project quality? How were they addressed?
Can this project be expanded in other geographic and programmatic areas based on need?
Were there any unintended multiplier effects negative and/or positive?
2.4 Accountability
Assess the effectiveness of the community feedback mechanisms in place and the capacity of Oxfam to respond to comments received? Was there a functional M&E system in place?
Were all stakeholders aware of the project, rational, targeting, duration and objectives?
How effective were the linkages between Oxfam UN house staff and Oxfam Juba staff especially linkages with support services, HR, Finance, Logistics with respect to adherence to internal processes and support?
Assess the upwards accountability to donors and the impact of and capacity to delivery on, donor reporting requirements.
Is there a monitoring and feedback system in place? How effective is it? How is data being used? Is there any evidence that feedback is influencing decision-making?
3.Evaluation Outputs
Based on the ToRs, agreement with Oxfam on the evaluation tools, methodology and detailed timeframe
Bullet point email progress updates
Draft evaluation report
Final evaluation report
Presentation of evaluation findings
Responsibilities: External evaluators to conduct the evaluation, EFSL team to assist with field facilitation/ data collection, UN House Field Coordinator and the Programme Quality Manager to act as the point of contacts for the external evaluators and ensure that evaluation deliverables are been met.
4.Timeframe and Duration
The final evaluation must be completed and approved by the 31st March 2015. Data collection must take place in February as Oxfam will no longer have a team present in the field sites from the 1st of March. It is anticipated the evaluation will take between 22-25 days.