Mid-Term Evaluation for the EU-funded Project in Tanzania, To certification and beyond: Market access for sustainable coffee, horticulture, and tea TZ Consultancy

Project Consortium

Tanzania Coffee Research Institute(TACRI): A Government research body with a mandate to initiate, implement, promote and carry out research on coffee production, processing, quality, farming systems, and husbandry of other crops associated with coffee. TaCRI has implemented three European Commission programs in the past and currently works with Solidaridad ECA to implement her Coffee Resilience Program in Northern Tanzania.

Tanzania Tea Research Institute (TRIT); A Government research body that supports the development of both small and large scale tea producers through appropriate, cost-effective, high-quality research and technology transfer, to ensure the sustainability of the Tea industry. TRIT is well versed in practical technological innovation for the tea sector.

Tengeru Horticultural Research and Training Institute (HORTI-Tengeru); HORTI-Tengeru was established with the national mandate to support horticultural activities connected with training, research, and extension of vegetable seed production and plant propagation. HORTI-Tengeru has experienced staff who extend training support to horticulture growers. Training implemented includes on-site, on-farm, and on-station training sessions.

Solidaridad Tanzania Country Office: Will carry out Country-level implementation roles for the PACE Project. The organization has been implementing programs in coffee, horticulture, sugarcane, cotton, dairy, gold, and landscape management in Tanzania. We have a rich experience in promoting best practices and sustainable production, trade, and market development for producers. Our programs in Tanzania focus on value chain development, food security, gender inclusivity, ICT, climate innovations, and investment in viable businesses for impact creation while working together with governments, multilateral agencies, private companies as well as commodity producers.

Project Background

Coffee, horticulture, and tea products are among the most important export agricultural commodities in Tanzania. Tanzania’s main export markets are India, Japan, China, Netherlands, Germany, United Kingdom, Kenya, Pakistan, and the United Arab Emirates. Coffee is Tanzania’s third-largest export crop contributing 14.1% of total export value. The production in 2016 totalled 59,502MT, worth US$ 173.6 million (55% Arabica and 45% Robusta). The main Arabica growing areas include North Kilimanjaro, Ruvuma, and Mbeya while those for Robusta include the Bukoba area of the Kagera region. The horticulture industry in Tanzania is the fastest growing sub-sector within the agricultural sector with an annual average growth of about 9 to 12% per annum. This record of growth is more than double the overall annual growth rate of the agricultural sector. In 2015, horticulture contributed 38% of the foreign income earned from the agriculture sector. The exports in 2015 reached US$ 545 million, compared to US$ 64 million in 2005. The Horticulture sub-sector employs about 2.5 million people, which makes the industry a major player in the country’s economic growth and employment sector. Tea contributes more than US$30 million to Tanzania’s export earnings, making it the fifth-largest export crop after cashews, coffee, cotton, and tobacco. The tea industry provides employment to 50,000 families and directly or indirectly benefits more than 2 million Tanzanians.

Global brands and retailers focus on sustainability as an integral part of their corporate social responsibility policies designed to ensure environmental, social, and economic welfare. In many ways, the rise of Voluntary Sustainability Standards (VSS) is a response to wider industrial and social transformation, and it has a growing impact on international trade. However, complying with VSS is not simple, especially for small and medium-sized farmers, as it takes great effort and investment in people, infrastructure, and institutions. Furthermore, the infrastructure and the public finance needed to support the compliance process do not exist. Often, this stops the majority of farmers from exporting to profitable markets in developed countries. Without a clear strategy and proactive support from governments and the international community, VSS runs the risk of reinforcing of the walls that already shut out small-scale producers.

Project Objectives

The overall objective of this action is to address issues surrounding the effective use of certification schemes and the use of voluntary sustainability standards (VSS) to improve and drive market opportunities for Tanzanian “branded” sustainable tea, coffee, and horticulture products. Solidaridad proposes to mobilize the value chain actors, including the government, research, private sector, and the VSS service providers to sensitize and create awareness of smallholder farmers on the sustainable production of high-quality agricultural commodities, food safety, benefits of VSS mechanisms, and eventual integration of sustainability practices in their business-as-usual model.

The Specific objectives will include: i) improving domestic, regional, and international market access for Tanzanian “branded” sustainable tea, coffee, and horticulture products; and ii) supporting the adoption of identified voluntary sustainability standards.

Project Coverage (geographic areas and number of beneficiaries)

The project targets approximately 21,000 smallholder farmers in Zanzibar (Horticulture), Mbeya, Njombe regions (Horticulture and Tea) and Ruvuma and Songwe regions (Coffee) in Tanzania.

Purpose of the Evaluation

The objectives of the MTE are to:

1) Assess performance towards the program’s outcome and output indicators so far and identify possible gaps towards achieving full results (accountability).

2) To capture lessons learned and provide information on the project approaches identifying what has worked and what has not worked for learning and as a guide for future planning and potential upscaling (learning)

Scope of Work and Key Tasks

The MTE will cover the implementation period from the start of the program. This MTE is expected to assess the effectiveness of the implementation strategy of the project in a consortium setup. This will include the implementation modalities and participation by the other consortium partners. The evaluation will include a review of the project design and assumptions made at the beginning of the program development process. It will assess whether results are on track, the implementation strategy has been optimum, and recommends areas for improvement and learning. The evaluation will also assess whether project resources are efficiently utilized to produce planned results and whether results can be realized within the planned period. Lastly, the sustainability of the project will be looked at to ensure a lasting impact.

Indicative Evaluation questions:

Relevance of the program strategy (design and focus of the program)

– To what extent are the program objectives meeting the needs of small-scale coffee, tea, and horticulture farmers and their families?

– Does the program address the key bottlenecks in accessing markets for the farmers?

– Are the underlying assumptions and context of the program still correct?

– Are the program results framework, indicators, and targets suitable (i.e. SMART) to monitor and support the implementation of the targeted results? If not, what are the suggested changes?

Effectiveness (project progress)

– To what extent has progress been made towards the planned outputs and outcomes, as defined in the program results framework? What recommendations for improvement can be made?

– Were there any unexpected positive or negative side effects and how were these dealt with by the program management?

– Are the chosen program interventions and activities effective in delivering the desired outcomes?

– To what extent was the program implementation effective (Management arrangements, work planning, finance and co-finance, project-level monitoring and evaluation systems, Stakeholder Engagement, reporting, communications)

– What factors have contributed to or hindered the achievement of intended outputs and outcomes?

– To what extent have partners in the consortium played their expected roles toward the achievement of the program goal?

– Has the consortium partnership been appropriate and effective?

Outcomes and Impact:

– How does the program contribute towards lasting change to the lives of beneficiaries with regard to:

– increased sustainable production of coffee, tea, and horticulture; if there would be no sustainable agricultural practices and production systems, what would the local production situation look like?

– Adoption of VSS and accessibility of markets; if there would be no adoption of VSS and market linkages, what would the situation look like?

– Has progress so far led to, or could in the future catalyze beneficial development effects (income generation, gender equality, and women’s empowerment, improved governance, etc.) that should be included in the project results framework and monitored on an annual basis for the remainder of the project?

Efficiency:

– Are the inputs identified realistic, appropriate and adequate for the achievement of the results?

– Do the actual or expected results (outputs and outcomes) justify the costs incurred? Are resources effectively utilized?

– What are the factors contributing to implementation efficiency?

Sustainability:

– Are there any remaining barriers to achieving the project objectives that require review?

– How can successful aspects of the project be further built to expand the benefits?

– What is the likelihood of continuation and sustainability of program outcomes and benefits after completion?

– What are the key factors that require attention to improve prospects of sustainability of outcomes?

– What are the important challenges the program should overcome? And to what extent are these adequately addressed?

– What are the key lessons learned based on the experiences of project implementation?

The Consultants may choose to categorize the questions differently, as long as these questions are addressed.

Methodology

This evaluation requires a mixed-method approach that allows for methodological triangulation to increase the validity and credibility of data. Participatory methods shall be used to collect both qualitative and quantitative data The key methods shall include the following but are not limited to:

● A household survey using a survey questionnaire (a representative sample of the total beneficiary population can be used)

● Focus Group Discussion (FGDs) with lead farmers and AMCOSs/farmer groups board members

● Literature review of existing documents and review of context and poverty trends including the project proposal and other documents, annual and quarterly reports, monitoring, and evaluation reports

● Field observation of the targeted project areas

● Key informant interviews (KII) with AMCOSs, farmer groups board members, Lead farmers, Extension officers, DAICOs, project partners (TACRI, TRIT, and Hort Tengeru), and project management team reflection and feedback sessions with the project management team.

The final methodology to be adopted shall be discussed and agreed upon with consultants/firm during the inception meeting.

The MTE team will first conduct a document review of project documents (i.e. Initiation Plan, Project Document, Project Inception Report, Baselines, Tracking Tools, Project Steering Committee meeting minutes, Financial and Administration guidelines used by Project Team, etc.) provided by the Project Team. Outcome and output indicators from the Logical Framework and reported upon in the baseline and annual reports will be used to the extent possible. In case these indicators are not sufficiently reported upon, data collection will need to be complemented.

The consultants will participate in an inception workshop to clarify their understanding of the objectives and methods of the MTE, producing the MTE inception report thereafter. The MTE mission will then consist of interviews and site visits to selected areas in the Project areas.

Expected Outputs & Deliverables

Solidaridad and the consultant will work together to identify the study team. However, Solidaridad recommends that consultants identify one Team Leader (TL) to coordinate/conduct the comprehensive study and liaise with Solidaridad. To collect information from the field, the consultant is expected to hire an adequate number of Field Enumerators (FEs), who will be deployed to the field in the facilitation of Solidaridad’s PMEL Officers. Supervision of the fieldwork and the quality (reliability and validity) of the data/information collected from the field is the primary responsibility of the TL. The TL will work closely with Solidaridad PMEL Officers and Project Team. In each step and process, consultation with the Program Manager and PMEL Officer is vital.

The consultancy/consultant team will be primarily responsible to:

1. Submit an expression of interest (EOI) for the study, outlining the proposed methodology and process of data collection and analysis that specifies:
   1. Proposed methodology for implementation of the study, including sample sizes.
   2. A detailed timeline for the study
2. Inception report based on the work plan and EOI:
   1. Detailed methodology and evaluation matrix
   2. Draft data collection tools for all indicators to be measured in the study, standard indicators definitions agreed upon by the project should be used to develop data collection tools.
3. Presentation of methodology to Solidaridad team in an inception meeting and incorporate suggestions provided.
4. Training of enumerators (Solidaridad PMs and PMEL Officers will be part of it)
5. Draft report of the study for the feedback and comments of Solidaridad and partners involved.
6. Presentation of the main findings of the study at field level validation involving project teams, local authorities, and other stakeholders as agreed with the project team.
7. A final report in English following the Guidelines under MTE Annex 1.
8. Datasets: The consultant shall submit all datasets, cleaned & raw, transcription or audio files during focus group discussions, KI, scripts for data analysis, and any form of data captured during the implementation of this assignment
9. The evaluator shall also submit a pager infographic summary of the finalized report.

Coordination and cooperation with Solidaridad

The Consultant undertaking the study will work under the coordination of PMEL Officer, who will support the mobilization of respondents and offer technical backstopping. The Project team will coordinate consortium partners in ensuring that the sampled survey respondents (beneficiaries and Key informants) are sensitized and mobilized for the exercise while availing a workable itinerary and technical backstopping.

Profile of the Evaluators

The MTE team will ideally consist of independent consultants that will conduct the MTE – one team leader (with experience and exposure to projects and evaluations in Tanzania) and local team experts.

The team must have strong combined expertise in the sectors of Certifications and Voluntary Sustainability Standards, agricultural economics, extension and education, gender, and Natural Resource Management. The successful applicant will propose a team that can provide adequate experience to conduct the assessment and that meets the following requirements:

1. Advanced degree (masters) in the area of agriculture, economics, agri-business development, social sciences, or other related disciplines;
2. At least five years of proven experience in leading and conducting similar exercises in Africa
3. Demonstrated experience in both quantitative and qualitative methods of data collection and analysis; including survey design, implementation of surveys and statistical data analysis, and the use of participatory appraisal techniques in data collection and analysis.
4. Ability to interpret and analyze complex qualitative and quantitative data and to present findings and recommendations in a clear and concise way;
5. Strong expertise in value chains (coffee, other food crops, horticulture)
6. Knowledge and sensitivity to the political and social context of Tanzania context is an asset;
7. Earlier experience with the work of Solidaridad is an asset.