Terms of Reference (TOR)

Mid-Term Evaluation

Building Resilience and Addressing Vulnerability to Pakistan’ (BRAVE) Programme

1. Purpose of the Evaluation

Concern Worldwide is an international, non-governmental, humanitarian organization dedicated to the reduction of suffering and working towards the ultimate elimination of extreme poverty in the world’s poorest countries. Concern has been working in Pakistan since 2001, when it initiated an emergency response program to address the Afghan refugee crisis in Balochistan. Concern later moved into emergency, early recovery, and long-term development programming in Balochistan, Khyber Pakhtunkhwa (KP), Punjab and Sindh. 

Concern Worldwide is implementing Community Resilience component of Building Resilience and Addressing Vulnerability to Emergencies (BRAVE) Pakistan Programme funded by Foreign, Commonwealth and Development Office (FCDO). Building Resilience and Addressing Vulnerability to Emergencies (BRAVE) is a five year and seven months (1 Jan 2023- 31 July 2028) Programme. The Programme is envisioned to carry out resilience and humanitarian response activities through a consortium. Core partners in the consortium for Phase I include Concern as the consortium lead, Agha Khan Foundation (AKF) implemented the Programme in districts of Astor, Ghizer in Gilgit Baltistan (GB), and Chitral in Khyber Pakhtunkhwa (KP), CESVI in district Charsadda and WHH in Nowshera in Khyber Pakhtunkhwa. To complement core partner expertise, three technical partners have been brought on board with specific roles: Help Age International is to enhance consortium’s strengths in gender and inclusion by providing technical assistance in delivering inclusive Climate change adaptation. Asian Disaster Preparedness Centre (ADPC) in designing and delivering trainings to consortium staff in CBDRM/Climate Change adaptation and provide support in key activities such as designing risk assessments, developing adaptation plan and establishing early warning systems. Overseas Development Institute (ODI) is supporting the consortium in developing and rolling out monitoring, evaluation and value for money (VfM) strategy. The Phase is formally concluded in these districts on December 31, 2025. The Phase II has been started in 5 additional districts including; Ghanche and Shigar in GB (AKF), Upper Dir (MH), Swat (WHH), and Shangla (CESVI) of KP. The current mid-term evaluation will focus on the Phase I target districts where the programme has been phased out as per the plan. 

The Programme is being implemented in two phases, beginning in January 2023. Phase 1 includes an nine-month inception period (Jan-Sept 2023), followed by 21 months (Oct 2023- June 2025) with addition of 6 months for suitability activities (i.e. December 2025) of implementation in 360 disaster prone villages in Gilgit Baltistan in Astor and Ghizer districts and Khyber-Pakhtunkhwa (KP), in district Chitral (Upper & Lower), Nowshera and Charsada selected for their vulnerability to climate induced disasters. As the consortium lead, Concern Worldwide, in collaboration with consortium members, is planning to conduct Mid-Term Evaluation (MTE) of the BRAVE Programme implementation (October 2023 to December 2025) in the Phase I target districts (Charsadda, Nowshera, Chitral in KP and Ghizer and Astore in GB) 

The primary purpose of the mid-term evaluation is to assess the program's progress toward achieving its intended outcomes and goals, providing an opportunity to identify key learnings and areas for improvement. By evaluating the effectiveness of current strategies and interventions, the mid-term evaluation helps to ensure that the program is on track to meet its objectives. It also facilitates adaptive management, enabling the incorporation of feedback to refine and enhance the program's approach for the remaining project period (Phase II). The, mid-term evaluations are designed to be evidence-based, participatory, and forward-looking, ensuring that insights gained are used to maximize the program's impact and sustainability.

2. Description of the Context

Pakistan is ranked 5th in the Global Climate Risk Index of countries most vulnerable to climate change. It is highly vulnerable to slow onset and extreme climate events, with both frequently disrupting formation of human and physical capital. Future climate trends in Pakistan will affect natural resources, water security, and amplify hazard exposure and vulnerability. 

Pakistan’s geography is a profound blend of northern and north-western high mountainous ranges, the plains of Indus Valley, the Valley, the coastal strip in the South and desert in the central and south-eastern regions, which represent the uniqueness of the country climatologically. Pakistan is a low-income country with a population of approximately 225 million and about 35% of the population is living below poverty line. In the past decade, the recurrent spells of extreme weather events such as floods, droughts, glacial lake outbursts, cyclones, and heat waves have taken a heavy toll on both life and property and adversely affected the country’s economic growth. Climate change raises the prospect that these and other natural hazards will surge in frequency and intensity in the coming decades—a stark aide-mémoire of Pakistan’s vulnerability to climate change impacts. The Intergovernmental Panel on Climate Change (IPCC) Sixth Assessment Report, August 2021 (AR6) for the Asia region transcripts that sensitivity to climate change threats, in agriculture-dependent economies (such as Pakistan), arises from their distinct geography, demographic trends, socioeconomic factors, and lack of adaptive capacity that when taken together, determine the vulnerability profile by perpetuating a vicious cycle of poverty. Addressing climate change alongside other environmental problems, while simultaneously supporting sustainable socioeconomic development, requires a holistic approach.

3. Description of the subject of the review/evaluation.  

Concern’s proposed Theory of Change (ToC) for Component One (Climate Resilience) of BRAVE is based on climate action planning, execution, and transfer of new applicable knowledge and practice.  The overarching impact of the BRAVE is envisaged as “extreme poor and vulnerable people in Pakistan are more resilient to climate-induced risks, shocks and stresses”, which will be achieved through expected outcome titled as “empowered resilient communities able to anticipate, adapt to, and absorb the negative impacts of climate hazards and risks”, to be achieved through the following five outputs: 

Output 1. Inclusive Climate Adaptation Forums (UC-CAF) formed which are implementing Climate Action Plans at different administrative level (Village and Union Council)

Output 2. Climate vulnerable communities and at-risk households are taking up more climate resilient and diversified livelihoods pathways

Output 3: Climate sensitive private sector pathways developed for priority actions that generate financing and jobs

Output 4: Climate sensitive public sector pathways for priority sector/s that generate climate knowledge, promote risk-sensitive practices including risk financing

Output 5:  Critical humanitarian relief and early recovery needs of the vulnerable disaster affected population are met in the target districts of Pakistan: 

4. Evaluation objectives and scope

Concern Worldwide is hiring the services of an experienced external consultant (firm or individuals) meeting the eligibility criteria on competitive process to conduct the external mid term evaluation of the BRAVE Programme. The overall purpose of this formative evaluation is to evaluate the project on DAC criteria of “relevance, effectiveness, impact, and sustainability”. The objective of this formative evaluation is to assess a program's progress and effectiveness during implementation, providing feedback to improve its design, processes, program strategies, and outcomes. The evaluation scope includes all five outputs/activities included in the project in implementation phase 1. The geographic focus includes all the five districts where the BRAVE activities were implemented in implementation phase 1.  

Furthermore, the Evaluator will carry out at least six outcome and impact studies/survey focus on a set of following key activities designed not only to achieve immediate results but also provide the outcomes and impact of these key interventions and generate relevant and actionable insights. This usually enables more relevant findings that can be used for future programme design and learning. 

1. Awareness Raising and communication on potential impacts of climate change and mitigation measures and Early Warning through Pakistan metrological Department (PMD)

2. Transformational Change – UCAF and VCAF

3. Assessment of the gender transformation activities including. 

• Training and leadership and negotiation Skills for VCAF/UCAF and community female activists.

• Training on Engagement with men for VCAF and UCAF male members

• Engagement with and capacity building of line departments for inclusive, gender-responsive CCA planning and service delivery

4. Piloting of climate-financing models and financial inclusion strategies particularly for women

5. Assess Signals of Transformational Change in VCAF and UCAF Using a Transformational Change Scorecard

6. Vocational skills - Tracer Study Report Compilation (there is enough data collected by respective consortium partner during the sustainability pahse, the evaluator will review and develop a comprehensive report and collect if there is need for any missing data) 

5. A - Evaluation Questions (DAC Criteria):

The key evaluation questions are mentioned below under each of the criteria included in the final evaluation. 

Relevance

• Were the programme’s objective and activities relevant to community priorities and needs in targeted villages?

• Are there instances of individual/household or community self-initiatives and mobilization as result of the programme? Is there any evidence that the ideas which the project has introduced have to spread to villages where the project has not worked?

Effectiveness: 

• To what extent were the planned outcomes achieved? Who has benefitted? 

• How satisfied are the stakeholders (communities, village councils, district government) with the programme interventions? What satisfied them most and why? What satisfied them less and why?

Impact:

• How have the programme interventions impacted differently on men and women, on people of different economic status, on people with disabilities (and other vulnerable groups as identified) in the programme area?

• Has there been a difference between the impact in highland and in lowland areas? Why have some people benefited more than others in the different districts?

• Did the programme have any unintended and unexpected consequences (positive and/or negative)? If so have these been addressed in the programme and how?

Sustainability

• To what extent are the benefits of the programme likely to continue after the completion of the programme? How are the climate adaptation forums and Hubs functioning independently, and how likely is it that they will continue to function after the support from BRAVE has needed? Why? 

Coordination/ Coherence 

• How well did the programme coordinate with Government line departments during design and implementation of programme? To what extent has this cooperation resulted in better programming? 

• Did the consortium model achieve consistency, where required, between members across its implementation modalities (harmonization of quality, quantity, targeting criteria, technical approaches etc.)?

Identify lessons to be learned to inform the future programme

• What are the key areas needing improvement in terms of design, approaches and implementation with particular emphasis on climate resilience?

5-B: Outcome/Impact Assessment Survey Questions:

1. Awareness Raising & Climate Communication (Climate Change Impacts, Mitigation Measures, and PMD Early Warning)

1. To what extent has awareness of climate change impacts and mitigation measures increased among target households and communities as a result of project-supported communication activities?

2. How well the EW Groups are working? How effectively have PMD early warning messages reached target communities, and how well are they understood and trusted by community members?

3. To what degree has increased awareness and access to early warning information led to changes in preparedness behaviours or livelihood decisions at household and community levels?

4. What differences, if any, are observed in response actions during climate shocks between communities with stronger exposure to project-supported awareness and early warning activities and those with weaker exposure?

II. Gender Transformation Activities Assessment

a) Training on Leadership and Negotiation Skills (VCAF/UCAF & Female Community Activists)

1. To what extent have the leadership and negotiation skills of female VCAF/UCAF members and community activists improved as a result of the trainings?

2. How has women’s participation and influence in community decision-making, planning, and climate adaptation forums changed over the project period?

3. What evidence exists of increased confidence, individual, or collective action among trained women in addressing climate and livelihood challenges?

4. Have these changes contributed to more gender-responsive community or household-level decisions? If yes, how?

b) Engagement with Men (VCAF/UCAF Male Members)

1. To what extent has engagement with men contributed to changes in attitudes and behaviours related to gender roles and women’s participation?

2. How have male VCAF/UCAF members supported or enabled women’s leadership and participation in climate adaptation and community structures?

3. What evidence exists of reduced resistance or increased cooperation from men in household or community decision-making processes?

4. How have these changes influenced gender relations and joint decision-making within households and community institutions?

c) Capacity Building of Line Departments (Gender-Responsive CCA)

1. To what extent have targeted line departments improved their capacity to plan and deliver inclusive, gender-responsive climate change adaptation (CCA) services?

2. What changes are observed in policies, plans, PC1 or service delivery practices as a result of project-supported capacity-building efforts?

3. How effectively are line departments ensuring the implementation of commitment they have made in the post training action plans?

4. What early outcomes suggest improved institutional responsiveness to gender and climate vulnerability at the local level?

III. Piloting Climate Financing Models & Financial Inclusion (with Focus on Women)

1. To what extent have pilot climate-financing models improved women’s and marginalized groups access to financial services, including savings, credit, or insurance?

2. How have these financial inclusion strategies contributed to women’s and marginalized groups economic resilience and ability to cope with climate-related shocks?

3. What changes are observed in women’s control over financial resources and decision-making at household and group levels?

4. Which financing models show the strongest potential for scalability and sustainability, and why?

IV. Transformational Change in VCAF and UCAF (Transformational Change Scorecard)

1. What evidence of transformational change (beyond incremental change) is observed in VCAF and UCAF structures over the project period?

2. To what extent have VC AFs and UC AFs shifted in terms of power dynamics, inclusiveness, gender equity, and accountability?

3. How effectively are VCAF and UCAF influencing local planning, service delivery, and climate adaptation decisions?

4. What factors have most enabled or constrained transformational change within these forums?

5. What is the current functionality status of the VCAF and UCAF? In terms of regularity and quality of meetings, record keeping and documentation, implementation and monitoring of Community Adaptation Plans, inclusiveness and participation (e.g., women and vulnerable groups), leadership and governance practices, resource mobilization, and effectiveness of coordination and linkages with government departments, service providers, and other stakeholders?

V. Vocational Skills – Tracer Study (Outcome & Impact Focus)

1. To what extent have vocational skills graduates achieved sustained employment, self-employment, or income generation following training?

2. To what extent does vocational training improve employment status, income levels, and livelihood stability of trainees compared to their pre-training situation or a comparable non-trained group?

3. How has participation in vocational skills training contributed to household income diversification and economic resilience, particularly during climate or economic shocks?

4. What differences are observed in outcomes for women and men, and what factors explain these differences?

5. How relevant and market-responsive are the vocational skills supported by the project, and what is their long-term sustainability potential?

6. What role should vocational training play within a broader livelihoods or resilience strategy, considering its cost, reach, and comparative impact on different population groups (e.g., youth, women, vulnerable households)?

6. Adoption of Climate-Smart Agriculture Practices to Enhance Crop Resilience Against Climate Change

1. What Climate-Smart Agriculture practices are being adopted by farmers, including the use of hermetic technology for seed storage, compost or organic manure, and Integrated Pest Management, and how are these applied in crop cultivation and post-harvest management?

2. To what extent do farmers continue using the promoted technology independently after initial project-supported use, and what factors influence their sustained adoption?

3. How do farmers perceive the impact of these practices on crop productivity, seed quality, and resilience to climate change?

4. What challenges or barriers do farmers face in adopting hermetic storage and other Climate-Smart Agriculture practices?

5. What sources of information, training, or support influence farmers’ decisions to adopt hermetic technology and other CSA practices?

5. Methodology

Evaluation methodology: includes both secondary data review and primary data collection using qualitative research methodologies. A review of secondary information including project/programme documentation such as Proposal, inception reports, baseline, end-line, theory of change, log frames, results frameworks, M&E plans, quarterly reports, annual reports, annual review, as well as more general relevant documentation such FCDO business case, climate change policies and country strategy. The qualitative data collection, including survey and focus groups discussions with beneficiaries and non beneficiaries; paying attention to different sub-groups (such as males and females), and different categories of beneficiary according to project activities. In addition, the evaluation must ensure that the views of the most vulnerable are included. The primary data collection should also include key informant interviews with community leaders, local government (district administration, Agriculture, livestock, irrigation, 1122, (provincial district level), consortium members, local/downstream partners, teachers, TVET, UCAF/VCAF, Knowledge/business Hub, and Concern staff. Field observation to assess the program implementation, and workshop/consultation o engage stakeholders in a participatory evaluation process, gathering feedback and validating findings. In addition to this, the evaluator can suggest any other method that can best fit to achieve the evaluation objectives.

Studies/Survey methodology:

In consultation with relevant BRAVE technical advisors and the M&E team, the consultant will identify key variables and indicators for each study. The consultant will develop the study methodology, including sampling and data collection approaches, and design appropriate data collection tools to generate reliable evidence for analysis and learning. The survey will use probability sampling method using statistically significant sample size to generalize the findings.

6. Expected products/outputs 

The evaluation will use a participatory approach for administering the evaluation and include the final deliverables.

1. Inception report – detail the evaluation methodologies, data collection tools, sampling design, list of respondents, secondary data and updated timeline of the evaluation (Maximum 15 pages excluding annexures) 

2. Final evaluation report (maximum 25 pages excluding annexures) along with standalone key summary report (not exceeding 3 pages)

3. Each study/Survey report

4. Reflection session on key evaluation findings with Concern and partner staff

1. Inception Report: the inception report will be submitted for approval prior to data collection, and should contain several key sections:

• List of secondary data and documents provided by Concern for review

• Methodology and sampling approach, including disaggregated sampling targets, and any anticipated limitations

• Updated work plan, including dates in which data collection will be conducted, timeline for preparation of report, and including persons responsible for managing listed tasks

• Data collection tools/templates

Note: The inception report should not exceed 8 pages of text, excluding annexures.

• Final Report: The evaluator(s) will produce/submit a report in hard and soft form along with relevant annexes (in Microsoft Word). The report should include:

• Executive summary

• Introduction and Project Overview

• Evaluation methodology and Limitations

• Findings and Discussion on specific Questions highlighted in the TOR under the relevant DAC criteria (Relevance, Effectiveness, Impact, Sustainability and Coordination) including findings and discussion related to the Cross Cutting Approaches. 

• Conclusions and Lessons Learned

• Recommendations and management responses

• Report Annexures: Final ToRs, evaluation schedule, Data Collection Tools/templates, list of persons interviewed and sites visited, documents consulted declaration of independence from the project team, and the updated program M&E Plan/Indicator Table with achievements against each indicator. 

• Study/Survey report: The evaluator(s) will produce/submit a report in hard and soft form along with relevant annexes (in Microsoft Word). The report should include:

• Executive summary

• Introduction and Project Overview

• Study methodology and Limitations

• Findings and Discussion on specific Questions highlighted in the tools 

• Conclusions and Lessons Learned

• Recommendations 

Report Annexures: Final ToRs, evaluation schedule, Data Collection Tools/templates, list of persons interviewed and sites visited, documents consulted declaration of independence from the project team, and the updated program M&E Plan/Indicator Table with achievements against each indicator

Note: The final report must have a stand-alone Executive Summary not exceeding on 3 pages. The main evaluation report (without annexes) will not exceed 25 pages to keep the audience straight to the main point.

7. Reporting lines  

The consultant will report to Concern’s Team Leader for BRAVE programme and will liaise closely with Senior/Technical Advisor MEAL, Concern’s HQ Advisor and other Programme Management Unit (PMU) Senior Technical Advisors (Resilience, Climate Change, Gender and Livelihoods). Apart from these regular meetings and interactions with the relevant team of Consortium partners will be made to discuss and finalize each output. In addition, if required, consultant may be asked to present his/ her work in Consortium Technical Working Groups for broader understanding and feedback of Consortium Partners. 

8. Composition, skills and experience of the Evaluation team

• Advanced University degree (at Master’s degree level or higher) in Development Studies, Climate Change, Disaster Risk Management, and/or related social sciences field. 

• Proven knowledge of the development sector particularly in the fields climate change, DRR, resilience, Infrastructure, Livelihoods and skills development in an integrated programme approach, as well as practical experience of Return on Investment, Value for Money analysis tools and techniques.

• At least 5-8 years of experience conducting evaluations of development projects with particular focus on resilience programmes using DAC criteria for evaluating development assistance.

• Knowledge of the Pakistani regional context and languages required within the team.

• Familiarity with international quality and accountability standards applied in emergencies;

• Experience in the use of participatory methodologies and developing gender sensitive evaluation methodologies.

• Excellent Qualitative and Quantitative data analysis skills with proficiency in relevant statistical software. 

• Excellent written and spoken communications skills in English. 

• Inter-personal and analytical skills with an extraordinary level of diplomacy and tact while dealing with numerous stakeholders.

• Excellent analytical, facilitation and reporting skills.

• Knowledge of the Government of Pakistan’s climate change and DRR policies and framreowrks and architecture.

While applying, please note that the evaluator will be assessed on the following areas:

• Understanding the context

• Technical approach

• Overall budget costs8 (once requested), and 

• Proposed evaluation team

Note: This consultancy is open to both national and international applicants. Qualified individuals, teams of individuals, and consulting firms with the required skills and relevant experience are eligible to apply. International consultants or firms must possess valid legal authorization to operate in the country and should have a competent national team capable of ensuring unrestricted field mobility and effective implementation of the assignment. National consultants must demonstrate the ability to travel freely and operate across all target districts.

9. Plan for Evaluation implementation (including timelines)

The evaluator will complete the work over a period of thirty (30) working days in the months of March and April at the date of signature of the contract and ending with the acceptance of the final report. Please note the start/end date of the assignment may change due to unavoidable circumstances in which case a revised timeframe will be drawn up with the mutual agreement of both parties. However, the evaluator will complete all the evaluation activities during the contract period. The evaluator will provide a detailed plan to complete this evaluation in the stipulated timeframe. Additional time (until 3rd week of April) can be provided for the finalization of report.

10. Applications

This consultancy is open to National applicants however international applicants having an existing Visa/NOC (allowed to work) to work in Pakistan are also encouraged to apply (in case of international consultant, s/he will be required to include a co-consultant (national) having experience of contextual understanding of Pakistan and the targeted districts in particular.  

The above-mentioned tasks to be performed are intended as a guide and should not be viewed as an inflexible specification as it may be modified following discussion with the selected consultant or firm.  

Interested consultants should send the following documentation in sealed envelopes to Plot#144, Street #30 | I&T Centre, Sector G-9/1 | Islamabad by/before February 28, 2026.

• Cover letter detailing the consultant’s/firms’ suitability for the assignment and current contact information

• A short description of methodology to undertake assignment and proposed budget inclusive of travelling, accommodation and food expenses etc. 

• Profiles/CVs of key person(s) to be involved at different steps during the assignments. 

• At least one relevant example of previous assignment of similar nature (one evaluation and one outcome assessment study) carried out preferably in Pakistan with international NGOs/UN agency (Concern will strictly ensure the confidentiality of the reports.)

• Only Technical proposals should be submitted in a sealed envelope. 

• The weights for evaluation of technical and financial aspects are 70% and 30% respectively. 

• Please clearly write consultancy title on left upper corner of outer envelope.

• Any queries related to this consultancy assignment can be directed to [email protected]well before deadline. Queries submitted after deadline are less likely to be responded. 

Note: 

• Consultancy companies shortlisted based on Technical proposals will be asked to submit their financial proposals.

• Please make sure that you have submit self-declaration (attachment available) of previous criminal conviction along with technical proposal as part of the application process. 

Safeguarding at Concern: Code of Conduct and its Associated Policies

Concern has an organisational Code of Conduct (CCoC) with three Associated Policies; the Programme Participant Protection Policy (P4), the Child Safeguarding Policy and the Anti-Trafficking in Persons Policy. These have been developed to ensure the maximum protection of programme participants from exploitation, and to clarify the responsibilities of Concern staff, consultants, visitors to the programme and partner organisation, and the standards of behaviour expected of them. In this context, staff have a responsibility to the organisation to strive for, and maintain, the highest standards in the day-to-day conduct in their workplace in accordance with Concern’s core values and mission. Any candidate offered a consultancy opportunity with Concern Worldwide will be expected to sign the Concern Code of Conduct and Associated Policies as an appendix to their consultancy contract. By signing the Concern Code of Conduct, candidates acknowledge that they have understood the content of both the Concern Code of Conduct and the Associated Policies and agree to conduct themselves in accordance with the provisions of these policies. Additionally, Concern is committed to the safeguarding and protection of vulnerable adults and children in our work. We will do everything possible to ensure that only those who are suitable to do consultancy work or volunteer with vulnerable adults and children are recruited by us for such roles. Subsequently, being hired as a consultant with Concern is subject to a range of vetting checks, including criminal background checking.