Endline Evaluation of the Strategic Partnership Agreement II Programme and Syria & Lebanon Top Ups

TERMS OF REFERENCE Endline Evaluation of the Strategic Partnership Agreement II Programme and Syria & Lebanon Top Ups Commissioned by | ActionAid - Arab Region Syria and Lebanon | 2022–2026 1. Background ActionAid Arab Region (AAAR), together with its local partners, has implemented the Strategic Partnership Agreement II funded by DANIDA in partnership with ActionAid Denmark in Syria and Lebanon from 2022 to 2026. The programme has been implemented in highly complex and protracted crisis contexts, requiring continuous adaptation to changing political, economic, humanitarian and protectioAn conditions. In Lebanon, implementation has been undertaken with SHIFT, Bassmeh & Zaitouneh, and RDFL, with interventions reaching communities in Tripoli, Jeb Jenine and Ba'albek. In Syria, implementation was initially concentrated in Northwest Syria, through Violet and Bahar, and subsequently expanded to Latakia and Damascus in response to changing contextual and humanitarian needs. The programme has also been complemented by two Top-Ups, one in Lebanon and one in Syria, which are included within the scope of this endline evaluation. The evaluation should therefore assess the overall contribution and performance of the core SPAII programme alongside the additional interventions and adaptations introduced through the Top-Ups. The programme as a whole is expected to have reached approximately: • 75,000+ people in Syria, including people benefiting from humanitarian assistance in its various forms, including distributions, protection services and legal services; people participating in trainings; people accessing youth and safe spaces; and people participating in campaigns, collective action and other forms of civic engagement. • 35,000+ people in Lebanon, through similar combinations of humanitarian assistance, protection and legal services, trainings, youth and safe spaces, campaigning and collective action. The programme has been structured around three Strategic Objectives: SO1: Rights and access to services Young people, especially marginalised young women, enjoy fundamental rights through improved access to well-resourced programmes and public services in fragile situations. SO2: Sustainable livelihoods and green solutions Young people from refugee and host communities, especially marginalised young women, enjoy improved, sustainable livelihoods through government that is accountable towards building sustainable green solutions. SO3: Resilience, protection and rights Young people from refugee and host communities, especially marginalised women, are resilient to shocks and enjoy their rights to protection in protracted crises. The programme's Theory of Change (ToC) provides the overarching logic through which interventions are expected to contribute to improved rights, access to services, livelihoods, resilience, protection, collective action and longer-term systemic change. Given the duration and complexity of the programme, the endline evaluation provides an opportunity not only to assess the extent to which the programme achieved its intended results, but also to examine how and why change occurred, the contribution of different interventions and partners, the quality of technical capacity and collective action, the programme's policy influence, and the extent to which programming remained relevant and adaptive in protracted crises. 2. Purpose of the Evaluation The primary purpose of the endline evaluation is to provide an independent assessment of the performance, contribution and learning of the SPAII programme and its Top-Ups in Syria and Lebanon between 2022 and 2026. The evaluation will assess the programme against the OECD DAC evaluation criteria of: 1. Relevance – Is the intervention doing the right things? 2. Coherence – How well does the intervention fit within its wider context and other interventions? 3. Effectiveness – Is the intervention achieving its intended objectives and results? 4. Efficiency – How well have resources been used to achieve results? 5. Impact – What difference has the intervention made? 6. Sustainability – Are the benefits, capacities, relationships and changes likely to continue? In addition, the evaluation will place particular emphasis on the programme’s ability to adapt to changing and protracted crisis contexts, the quality and adequacy of ActionAid/AAAR’s technical leadership, guidance and support, the effectiveness and quality of collective action, the extent and nature of policy influence, and the relationship between programme implementation and the Theory of Change. The evaluation is expected to generate practical, forward-looking evidence and learning to inform future AAAR programming in Syria and Lebanon, the design of regional and country-level programmes, partnership and localisation approaches, adaptive programming in protracted and rapidly changing crises, technical leadership and support to partners, advocacy and policy influencing, youth engagement and collective action, and humanitarian-development-peace programming. Ultimately, the findings will contribute to AAAR’s organisational learning and strategic decision-making, helping to identify what worked well, where the programme could have done better, why particular approaches succeeded or underperformed, and what should be sustained, adapted or strengthened in future programming. 3. Evaluation Questions The following questions should guide the evaluation. The evaluator should refine and prioritise these during the inception phase. A. Relevance 1. To what extent did SPAII respond to the priority needs, rights and aspirations of young people, particularly marginalised young women? 2. To what extent did interventions remain relevant as the contexts in Syria and Lebanon changed between 2022 and 2026? 3. How well did the programme balance immediate humanitarian needs with longer-term rights, resilience, livelihoods and systemic change? 4. Were there important needs or groups that the programme did not adequately address? B. Coherence 1. How well did SPAII complement other interventions implemented by AAAR and partners? 2. To what extent did the programme create meaningful links between humanitarian response, protection, rights, livelihoods, resilience and advocacy? 3. How well did interventions align with local systems, priorities and existing services? C. Effectiveness 1. To what extent did SPAII achieve its intended outputs and outcomes? 2. Which interventions and approaches contributed most significantly to results? 3. Which interventions were less effective, and why? 4. To what extent did the programme achieve meaningful changes beyond activity and output-level results? 5. What factors enabled or constrained effectiveness? 6. Were there significant differences in results across countries, locations, partners, groups or types of intervention? D. Theory of Change 1. To what extent did the programme's Theory of Change accurately explain how change occurred? 2. Which pathways of change were strongest? 3. Were there unexpected pathways or outcomes? 4. What contextual factors influenced the pathways of change? 5. What should be retained, modified or removed from the ToC for future programming? E. Efficiency 1. Were resources appropriately allocated across interventions and locations? 2. To what extent did the programme achieve value for money? 3. Were programme resources sufficiently flexible to respond to changing needs? 4. How did partnership arrangements affect efficiency? 5. Were there significant operational or coordination constraints affecting implementation? F. Impact 1. What meaningful changes can be attributed or plausibly contributed to by SPAII? 2. What difference did the programme make to young people's rights, resilience, protection, livelihoods and participation? 3. What unintended positive or negative effects emerged? 4. To what extent did the programme contribute to changes beyond direct beneficiaries? G. Technical leadership and guidance 1. To what extent did AAAR provide adequate technical leadership and guidance to support the quality and effectiveness of SPAII implementation? 2. How well did AAAR's technical support respond to the specific realities of Syria and Lebanon, including the constraints associated with protracted crisis contexts? 3. Where were technical leadership or guidance gaps identified, and how did these affect implementation or results? 4. What technical guidance, systems or approaches should be retained, strengthened or developed for future programming? H. Collective Action and Policy Influence 1. What was the quality and significance of collective action supported by SPAII? 2. To what extent did young people influence decisions, institutions, policies or public discourse? 3. What evidence exists of policy or systems-level influence? 4. What enabled or constrained meaningful youth participation? 5. Which collective action approaches should be continued or scaled? I. Adaptation in Protracted Crises 1. How effectively did the programme adapt to changing political, economic, humanitarian and social conditions? 2. Which adaptations were most significant? Were adaptations timely and appropriate? 3. What enabled effective adaptation? What constrained adaptation? 4. Did adaptations maintain alignment with the programme's strategic objectives and ToC? 5. What does SPAII demonstrate about how programmes should be designed and managed in protracted crises? J. Sustainability 1. What programme benefits are likely to continue after funding ends? 2. Which capacities, systems, relationships and practices are likely to be sustained? 3. To what extent have local partners been positioned to continue the work? 4. Are collective action and youth engagement mechanisms likely to continue? 5. What institutional or contextual risks threaten sustainability? 6. What should AAAR and partners do to strengthen sustainability? 4. Evaluation Scope The evaluation will cover: Lebanon: Tripoli, Jeb Jenine, Ba'albek Syria: Northwest Syria, Latakia, Damascus The evaluation should ensure adequate representation of different geographic and contextual settings rather than relying solely on the most accessible locations. Time period 2022–2026 Programme scope The evaluation will cover: • Core SPAII interventions; • all Top-Ups; • all three Strategic Objectives including: humanitarian assistance; protection services; legal services; trainings and capacity strengthening; youth and safe spaces; livelihoods-related interventions; campaigns and collective action; advocacy and policy influence; partnership and localisation approaches; programme adaptation and learning. Partners Lebanon: SHIFT, Bassmeh & Zaitouneh RDFL Syria: Violet, Bahar 5. Evaluation Approach and Methodology The evaluation should use a mixed-methods, participatory and theory-based approach. A purely indicator-based assessment will not be sufficient. The evaluation should combine quantitative evidence with qualitative investigation of contribution, change, adaptation and learning. The evaluator is expected to propose a detailed methodology during inception. This should include, as appropriate: 5.1 Document review The evaluation will include a review of key programme documentation, including the programme design and Theory of Change, logframes and IPTTs, monitoring and DQA data, partner and monitoring reports, assessments and previous evaluations, learning and advocacy documentation, Top-Up documentation, and relevant contextual and needs assessments. 5.2 Quantitative analysis The evaluation will analyse available monitoring data to assess programme reach, participation, demographic and geographic coverage, achievement against targets, and trends across the implementation period and intervention types. The evaluator should critically assess data quality and limitations and should not treat reported reach as equivalent to meaningful change. 5.3 Qualitative research The evaluation will use qualitative methods to capture the perspectives of programme participants, young people and marginalised young women, communities, partners, volunteers, AAAR staff, technical teams, duty bearers, service providers, civil society and policy stakeholders. Methods may include key informant interviews, focus group discussions, participatory reflection, outcome harvesting, contribution analysis and selected case studies, as appropriate. 5.4 Theory of Change assessment The evaluator should conduct a dedicated assessment of the SPAII ToC, identifying: Inputs → Activities → Outputs → Outcomes → Strategic Objectives → Longer-term change while testing the assumptions and contextual conditions between each stage. The evaluation should identify successful and unsuccessful pathways of change and explain why they occurred. 5.5 Case studies The evaluator will develop a limited number of illustrative case studies highlighting significant examples of individual or community change, collective action, policy influence, technical leadership and guidance, adaptation to crisis, innovative or alternative solutions, and unintended change. Case studies should focus on the pathways and mechanisms of change, demonstrating how and why change occurred rather than simply describing activities or outputs. 6. Partnership and Localisation The evaluation should assess the partnership model as an important determinant of programme performance. Questions should include: • Did partners have sufficient ownership and decision-making space? • Was technical accompaniment appropriate? • Did AAAR add value beyond financial support? • How effective was knowledge transfer between AAAR and partners? • Were roles and responsibilities clear? • Did partnership arrangements support adaptation? • How sustainable are capacities developed through the programme? • What could AAAR do differently to strengthen locally led programming? The evaluation should particularly examine the balance between partner autonomy and AAAR technical accompaniment, quality assurance and strategic support. \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ 7. Ethics and Safeguarding Given the sensitive nature of programming in Syria and Lebanon, the evaluation must adhere to ActionAid safeguarding requirements and ethical research principles, including informed consent, confidentiality, anonymity, data protection, do-no-harm and conflict-sensitive approaches, gender-sensitive research, and safe participation of young people. Appropriate referral procedures should be in place where sensitive protection concerns arise. The evaluation should minimise risks associated with discussing protection, legal assistance, displacement, conflict and other sensitive experiences. No personally identifiable information should be included in the final report unless explicitly authorised and ethically justified. 8. Deliverables The selected evaluator/team will be expected to produce: I. Inception Report Including: refined evaluation questions; evaluation matrix; detailed methodology; sampling strategy; data collection tools; analysis framework; ethical and safeguarding approach; workplan; limitations and mitigation measures. II. Data Collection Tools Including appropriate tools for: KIIs; FGDs; participant consultations; partner consultations; policy stakeholder interviews; case studies; and other proposed methods. III. Preliminary Findings Presentation A presentation of emerging findings to AAAR and relevant partners for validation and reflection. IV. Draft Evaluation Report Including: • executive summary; • programme background; • methodology; • limitations; • findings; • conclusions; • lessons learned; • recommendations; • annexes; • and evidence supporting key findings. V. Final Evaluation Report Maximum recommended length: 40–50 pages excluding annexes. VI. Learning Product A concise SPAII Learning Brief synthesising key lessons and practical recommendations, including what worked and what did not, why, what AAAR and partners should continue or do differently, and what should be adapted to strengthen future programming in protracted-crisis contexts. VII. Final Learning Workshop Facilitation of a participatory learning session with AAAR and relevant partners to validate findings, identify key lessons and priorities, translate findings into actionable recommendations, and discuss implications for future programming. 9. Sampling The evaluator should propose a purposive sampling strategy that ensures representation across: • Syria and Lebanon; • geographic locations; • partners; • Strategic Objectives; • intervention types; • gender; • age groups; • marginalised groups; • refugees and host communities where applicable; • direct participants and wider stakeholders. The evaluation should avoid relying exclusively on programme success stories or easily accessible participants. 10. Key Evaluation Principles The evaluation should be: Independent – providing an impartial assessment of programme performance. Utilisation-focused – producing findings that can directly inform future decisions. Participatory – ensuring meaningful engagement of partners and programme participants. Theory-based – testing the programme's Theory of Change rather than simply measuring indicator achievement. Contribution-focused – recognising that change in Syria and Lebanon cannot necessarily be attributed to SPAII alone. Context-sensitive – recognising the complexity of protracted crises. Gender-responsive and youth-centred – examining who benefited, who did not, and how power and inclusion influenced results. Learning-oriented – identifying not only successes and failures but the reasons behind them. 11. Key Expected Evaluation Outputs Ultimately, the evaluation should answer five practical questions for AAAR: A. Where did SPAII get it right? What approaches, interventions, partnerships and adaptations generated meaningful results? B. Where did SPAII have an opportunity to do better? What underperformed, and why? C. What actually contributed to change? Which pathways in the Theory of Change worked, for whom and under what conditions? D. What did SPAII learn about programming in protracted crises? How can AAAR become more adaptive, locally led and responsive? E. What should AAAR do differently next time? What should be continued, stopped, adapted, strengthened, scaled or tested differently? 12. Timeline Activity Timeline Inception phase Week 1 Data collection Week 2 to Week 6 Data analysis Week 7 Validation workshop Week 8 Draft final report Week 9 Final report and learning product Week 10 Final workshop Week 11 13. Budget Maximum 16,230 JODs inclusive of 5% tax for Jordanians and 10% for non-Jordanians. 14. Required Qualifications of the Consultant/Team • At least 7–10 years of experience in evaluation of multi-year development programmes • Proven experience in youth-led programming, gender justice, climate resilience, and/or humanitarian programming • Strong knowledge of Jordan context, refugee/host-community dynamics • Demonstrated experience in designing exit strategies or sustainability assessments • Strong MEL background, including OECD-DAC evaluations is preferrable • Experience assessing advocacy and policy-influencing results • Arabic and English fluency • Experience working with grassroots organizations and youth structures 15. Management & Coordination • The consultant will report to the AAAR PQMEAL Coordinator and SPA II Programme Manager. • Local partners in Lebanon and Syria will facilitate, youth mobilization, and access to field sites. • AAAR will provide project documentation, MEL data, and monitoring tools. 16. Ethical Considerations The assessment must follow ActionAid’s safeguarding principles and ethical standards: • Do-no-harm approach • Informed consent, confidentiality, secure data handling • Gender-sensitive and inclusive methodologies • Special protection for youth under 18 and refugees