Consultant for External Evaluation of BMZ Phase I EiE
Terms of Reference – Evaluations
This standard Terms of Reference (ToR) for contracts to evaluators provides guidance for evaluations of private agencies co-financed by the BMZ.
Evaluation: BMZ-supported community-based education (CBE) project: Improved access to quality and inclusive basic education for children from returnee, IDPs and host communities in Kunduz province
1. Background and Rationale
- Brief description of the (initial) situation in the country/region/sector as well as the project activities and its impact logic, including its previous history and planned continuation, if applicable
- Rationale of the evaluation (concrete reason and need); presentation of the result of preparatory work/preliminary analyses to which the evaluation is linked
The Afghanistan education system has been seriously affected by the prolonged conflicts over the past several decades. In the community-based education (CBE) policy guidelines of 2012, the Afghan government estimates that over 3 million school-aged children have never enrolled in school while around one million are permanently absent. Historically, sanctions against girls’ education, attacks on formal schools, general insecurity, and shortage of qualified and especially female teachers combined with low number and long distances to schools has had negative impacts on children’s access to schooling in Afghanistan. The volatile security situation and recurring internal displacements as well as waves of returnees from Pakistan and Iran have further exacerbated the situation.
Kunduz province, and the target district Qala-e-Zal in particular, is one of the most conflict-affected regions in the country and is approximately 98% controlled by non-government actors, with only the center of each district under government control. Kunduz has welcomed a large number of internally displaced persons (IDPs) and returnees due to the ongoing conflicts between government armed forces and armed opposition groups (AOGs). Subsequently, the number of children who need basic education has increased significantly since 2016 due to these clashes and the influx of returnees from Pakistan and Iran. While the formal education system in Kunduz is already facing major difficulties in providing basic access to education to children of host communities, it does not at all have the capacity to provide education opportunities for children of IPDs and returnee populations. Considering the increasing demand for education and the low capacity of the public education sector, CBE has been proven as a recognized and effective solution to provide access to quality education to children in remote and hard to reach areas.
To address the major structural barriers to education in Kunduz province, SC and its local partner Coordination for Afghan Relief (CoAR) implemented a two-year (2018-2020) CBE project to provide quality basic education to conflict-affected IDP, returnee and vulnerable host community children aged six to nine years old by establishing 40 CBE classes in the target districts of Qala-e-Zal and Kunduz Centre. In line with SC best practices and CBE model in Afghanistan, the BMZ CBE project design is outlined below.
Phase I Results Framework
Overall goal (Impact): Improved access to quality and integrative basic education for returnee, IDP and host community children in Kunduz province
Project goal: 1,400 children (55% girls) have ccess to 40 CBE classes in Kunduz and benefit from quality, community-based education and improved education outcomes
IDP, returnee and vulnerable host community children have access to well-equipped educational facilities in Kunduz province
IDP, returnee and vulnerable host community children have access to quality education services and teaching
The capacity and motivation of local communities to support education and ensure access for children is strengthened
Partner capacities are built up so that the local partner is able to carry out CBE projects independently according to international standards
In September 2020, a second phase of the BMZ CBE project is planned to begin, which expands the Phase I project design and targets and is informed by lessons learned and a feasibility study conducted by an external consultant.
2. Purpose, Objectives and Use
- Where appropriate, reference to specific further activities into which the results of the evaluation will be fed
The primary objective of the project evaluation is to assess the efficiency and effectiveness of the BMZ-funded CBE project in supporting IDP, returnee, and vulnerable host community girls and boys to gain improved access to quality basic education in Kunduz province of Afghanistan. This evaluation will assess progress made by the project in providing increased access to quality community-based education and in improving the learning outcomes of enrolled children.
The outcome of this evaluation will inform both donor and implementer(s) of the performance of the project’s design and implementation against the OECD DAC criteria to provide suggestions for ongoing and future (community-based) education programming, specifically for adapting and strengthening Phase II of the BMZ-funded activity.
3. Scope of Work
- Temporal and geographical scope of the evaluation, (sub)activities to be specifically examined
- Central evaluation questions, criteria, and, if applicable, indicators to be processed or answered
- Methodical references
The project evaluation should consider assessing the expected outcomes defined in the project proposal by measuring progress made on the indicators defined in the logical framework. Furthermore, the different components defined in the evaluation questions (refer to Table A) should be assessed. The project evaluation should assess the different components (access, quality, community mobilization, capacity building) of the project within Qala-e-Zal and Kunduz Centre districts of Kunduz province where the project has been implemented since August 2018.
Table A – Evaluation Questions
Criteria Secondary Questions
Relevance Are the objectives of the project consistent with beneficiaries’ needs and priorities? *
How important is the relevance or significance of the intervention regarding local and national requirements and priorities? *
Acceptable and appropriate Is the intended program/project acceptable to the local community and stakeholders? Will they be willing to participate and engage?
Is the intended program/project appropriate to this particular context?
Fidelity Was the program/project implemented as intended?
Did implementation meet quality standards / best practice?
What were the barriers and facilitators to implementation?
Process How can implementation of the program be improved?
Reach and uptake Did the program/project reach its intended target populations?
What proportion of eligible/referred beneficiaries engaged in the program?
How do children/families who accessed and completed the program differ from those who did not (including high absenteeism or drop-out)?
What were the barriers and facilitators to beneficiaries accessing/
completing the program?
Satisfaction and experience How satisfied were the program/project beneficiaries (disaggregated by gender, age, teachers, parents, community stakeholders)?
Did program beneficiaries feel the services they received were acceptable, appropriate, and suited to their needs (disaggregated by gender, age, teachers, parents, community stakeholders)?
Impact What is the impact or effect of the programme or project in proportion to the overall situation of the target group or those affected? *
What change did the previously out-of-school children feel or experience as a result of enrolling in a CBE? (disaggregated by gender)
What change did the parents of previously out-of-school children (especially girls) experience as a result of enrolling their children in a CBE (incl. beliefs, negative coping mechanisms)?
Effectiveness Did the program/project achieve its intended outcomes?
Did the targeted children feel that they received quality education? If not, why not, if yes, why?
Are there any differences in outcomes achieved by different groups (disaggregated by gender, disability)?
Were there any unintended outcomes?
What were the driving/enabling factors for parents to enrol children (esp. girls) in school? (incl. well-equipped education facility, beliefs/views of stakeholders outside family)
Was there anything that would have made parents feel more comfortable to send their children (especially girls) to school before and after the project?
Equity and equality Is there evidence that the intervention reduces inequality and marginalization for specific groups?
Sustainability How is the sustainability or permanence of the intervention and its effects? *
How sustainable is the CBE model in this particular context? What are the barriers and facilitators? (including community ownership for (girls’) education, capacity of hub schools)
Cost-benefit What was the ratio of costs to benefits (cost-benefit analysis)?
Cost effectiveness Has the intervention been cost-effective (compared to alternatives)? (cost effectiveness)
Efficiency Was the programme or project implemented in the most efficient way?*
Replicability and scale Are the evaluation findings generalizable to other contexts?
Can the program/project be scaled up at cost?
Gender sensitivity How has the program/project considered gender sensitivity both in the design and its implementation of activities?
Has the program/project incorporated different needs and accessibility of boys and girls, men and women?
What are the gender gaps that the program/project addressed and what remaining aspects need to be considered further?
Child participation How were children meaningfully involved in the program/project?
In what ways would children have liked to be more involved?
Child safe programming Has the program/project been designed, planned, implemented and monitored to ensure it is safe for children?
How has child safety been integrated into the program/project design and implementation of activities? What aspects of the program/project make children feel safe?
Accountability How has the program/project approached accountability to children?
*OECD DAC Criteria
- Description of the phases of the evaluation, the activities and the actors involved
The following activities will be carried out by the consultant to conduct the evaluation:
Phase 1: Orientation
Meet with Sr. Program Manager/Coordinator(s), Technical Advisor, and MEAL focal point to introduce the project and discuss process for evaluation
Complete desk review of project documents shared by SC staff
Phase 2: Preparation
Develop methodology, sampling strategy, tools and materials for evaluation
Present approach to SC staff
Train field data collection team/enumerators
Phase 3: Implementation
Conduct meetings, interviews, and FGDs per planned schedule
Coach data collection team and troubleshoot issues that arise during data collection
Phase 4: Analysis
Clean and analyze the data
Write data analysis and present findings to SC staff
Finalize the evaluation report
5. Outputs and Deliverables
- Listing and further explanation of all products and services to be provided by the contractor in the course of the evaluation
- Determination of the working and reporting language and references to necessary translations
- Inception report following desk review of project documents and supplementary research (includes draft details of methodology, tools and timeline/ work plan for the evaluation, data collection and analysis tools, qualitative and quantitative protocols for data collection and analysis to be approved by SC Technical Advisor prior to field work
- Final tools, methodology and sampling strategy, and training materials for enumerators
- 3-page summary of project evaluation findings
- Presentation to SC staff and relevant stakeholders on evaluation findings
- Draft and final evaluation reports (maximum 25 pages excluding annexes)
6. Expert profile of the Evaluation Team
- In case of commissioning a consulting agency or a consulting institute: type of assignment and description of the required technical, organizational and other capacities
- Outline of the requirement profile of the evaluators
- Description of the evaluation team and its composition; in the case of individual assignment also of the different roles and tasks in the team, possibly differentiated according to the different phases of the evaluation
- Statement of the independence of the evaluators
The evaluator should have:
- previous experience working with humanitarian/development organization
- previous experience undertaking evaluations
- previous experience in engaging with education projects highly desirable
- excellent written and spoken English
- ability to train enumerators or hold feedback sessions which are in depth but do no harm
- clear understanding of Child Safeguarding
- previous experience working with children during prior assignments important
7. Tentative Timetable
- Detailed presentation of milestones and schedules from the first activity to the end of the contract, including interim results where applicable
No. Activity Deliverable (Y/N) Expected Delivery
1 Introductory meeting with SC team N 7 June
2 Desk review of provided materials and supplementary research completed N 10 June
3 Inception report submitted Y 14 June
4 Presentation of sampling strategy, methodology and draft tools to SC team N 16 June
5 Final tools, methodology and sampling strategy and training materials Y 18 June
6 Train field data collection team/enumerators N 21 June
7 Conduct interviews and FGDs N 21-24 June
8 Clean and analyze data N 25-28 June
9 3-page summary of data analysis submitted Y 30 June
10 Presentation of findings to SC team Y 2 July
11 Revise final evaluation report N 3-4 July
12 Final evaluation report submitted Y 5 July
8. Management of the Evaluation
- Brief description of the respective roles and functions of those involved in the management of the evaluation and of the stakeholders to be involved in the evaluation
The consultant-led evaluation will be managed by the MEAL Team in-country in Afghanistan with technical oversight from the Education in Emergencies Advisor. All deliverables will be approved by the Education in Emergencies Advisor and Director of Programme Development, Quality, and Advocacy (PDQA) in Kabul. The Programme Management team, inclusive of the local partner CoAR, will be responsible for addressing the findings of the evaluation outlined in the final report.
9. Quantity structure
- List in tabular form of the various activities and the number of working days per evaluator
To be determined based on the proposal submitted by the candidates.
- Template Report Evaluation BMZ Bengo EN
- Phase I proposal and latest programmatic report
- SC Child Safeguarding Policy and PSEA Policy
- Enumerator training guide (how to interview children who have experienced trauma)
Date advertised: 29 Jun 2020
Closing date: 11 Jul 2020 - 05:59 BST
Location: Kabul, Afghanistan
Department: Programme, Development and Quality
Type: Fixed-term contractSchedule: Full-time