Position description
International Orthodox Christian Charity (IOCC) in partnership with Diakonie Katastrophenhilfe launches the terms of reference for the external evaluation for the multi-year project “Rehabilitation and expansion of infrastructure and basic services in the health sector in order to improve the quality and access of health care for refugees and vulnerable host communities”, implemented in Lebanon, and funded by the German Federal Ministry of Economic Cooperation and Development (BMZ) under the funding channel of the Transitional Development Aid.
In response to the dire circumstances that Lebanese residents are suffering from, DKH, in partnership with IOCC proposed a project in 2021 for a period of four consecutive years.
Description of the project
This project focuses on transforming eight dispensaries into sustainable green buildings, enhancing accreditation procedures to join the Primary Health Care Centers (PHCC) network, and improving access to essential health services for vulnerable individuals. The project implemented energy-efficient systems, sustainable building materials, and green spaces to reduce the environmental footprint of the dispensaries.
Concurrently, comprehensive training and technical support is being provided to health staff and management to meet and maintain accreditation standards, ensuring high-quality health services.
To address the needs of vulnerable populations, the project offers subsidized health consultations and diagnostic tests, reducing financial barriers to care. Additionally, health awareness sessions are organized, and psychosocial support (PSS) services are provided to promote mental well-being and education.
Through these integrated activities, the project aims to create a sustainable, high-quality, and accessible primary health care system that benefits vulnerable Syrian and Lebanese families. The project is focused on reaching the following subcomponents:
- The health systems of the primary health care centers in the project region are improved through trainings and on-the-job coaching of health staff and management as well as the initiated accreditations for eight dispensaries with the aim of improving the quality of health services for vulnerable Syrian and Lebanese families in the project regions.
- The ecological, economic, and social sustainability and access for people with disabilities in the primary health care centers is improved through innovative and environmentally friendly ‘’green” solutions and inclusive access.
- The access to healthcare centers was made available for both refugees and host communities, limiting the competition over services and therefore leading to enhanced social cohesion.
The project activities are implemented in the following locations:
Location - Governorate - Region:
- Ousrati Health Center - Akkar - Deir Dalloum
- Abboudieh Health Medical Center - Akkar - Abboudieh
- Tripoli Social Health Center - Tripoli - Tripoli
- Mar Francis PHC - Zahle - Zahle
- Mashael Medical Center - Zahle - Al Omarieh
- Bekaa Medical Village - Zahle - Bar Elias
- CPMI - Beirut - Ashrafieh
- Al Islah - Beirut - Tarik El Jdide
Target Beneficiaries:
IOCC engaged members of the targeted communities in designing project activities to ensure interventions effectively addressed their needs, particularly access to quality and affordable healthcare services. Initial assessments, including FGDs and consultations with healthcare center directors, highlighted increased demand for healthcare services among both Lebanese and Syrian populations due to the economic crisis. Healthcare centers faced significant challenges, including limited capacity, high operational costs, and difficulties in retaining medical staff and procuring essential equipment, leading to service disruptions and heightened community tensions.
The project specifically targeted Syrian refugees and vulnerable Lebanese across various demographics, prioritizing children, pregnant and lactating women, the elderly, and people with disabilities. IOCC collaborated with healthcare centers to establish vulnerability criteria, ensuring services reached the most in need. Service provision varied based on each center’s capacity, staffing, and available resources.
The Participatory Community Impact Assessment (PCIA) emphasized the importance of reviewing criteria and processes based on beneficiary and non-beneficiary feedback. Pre-approved vulnerability criteria were incorporated into agreements between IOCC and healthcare centers, with defined roles and verification processes to ensure accountability.
Additionally, the project engaged men through Multi-Stakeholder Groups (MSGs), FGDs, community mobilization, health awareness sessions, and individual counseling, promoting a gender-sensitive approach and increasing awareness of maternal and child health and care.
The outcomes below provide a general overview for the intended achievements of the project:
Outcome 1: The health systems of the primary health care centers in the project region are improved through trainings and on-the-job-coaching of health staff and management as well as the initiated accreditations of the dispensaries with the aim of improving the quality of health services for vulnerable Syrian and Lebanese families in the project regions.
Outcome 2: Ecological, economic, and social sustainability and access for people with disabilities in the primary health care centers is improved through innovative and environmentally friendly ‘green’ solutions and inclusive access.
Outcome 3: Equal access to basic health services and improvement of the health situation of vulnerable Syrian and Lebanese families, including of pregnant and lactating women and their children in the project areas, through improved accessibility of services in the health centers.
Objectives of the Evaluation
The external evaluation intends to assess the impact of the project activities throughout the full period of the project implementation.
Scope of the Assignment
Stakeholders to participate in and benefit from the assignment
Geographical scope of the assignment:
- Evaluation of the project activities: Implementation areas: Bekaa, Beirut, North Lebanon, and Akkar
- Gender Impact Assessment: Bekaa, Beirut, North Lebanon, and Akkar
- PCIA: Bekaa, Beirut, North Lebanon, and Akkar
Target Audience of the Assignment
- DKH and IOCC project staff and organizational leadership
- The directors of the dispensaries
- BMZ program staff
- Ministry of Public Health
- Ministry of Environment
- Ministry of Energy and Water
- Nutrition, Health, and WASH Clusters
Evaluation Criteria
The evaluation will adopt the OECD/DAC evaluation criteria, including Relevance, Appropriateness, Efficiency, Effectiveness, Impact, and Sustainability, to provide a comprehensive overview of the project across all targeted locations and communities. Additionally, the evaluation will include assessments of gender impact and Peace and Conflict Impact Assessment (PCIA), as applicable to the project's scope.
Specific questions in relation to the set criteria include:
Efficiency
- To what extent did the greening component contribute to lowering the centers’ operational costs?
- To what extent did the on-the-job coaching training facilitate the accreditation process?
- Did the centers receive sufficient support from the project? If yes, what are the good practices to replicate, and if not, what were the main limitations?
Effectiveness
- Did the project achieve its intended goals and objectives?
- How successful was the project in addressing the identified needs or problems?
- To what extent was the Complaint Response Mechanism accessible to all community members, including various gender and age groups and to what extent was the CRM system functional and appropriate?
Relevance and Appropriateness
- To what extent were the project’s objectives and activities aligned with the needs and priorities of the target beneficiaries (men, women, elderly, children, and people with disabilities)?
- How well did the project respond to the current situation or context (e.g., socio-political, economic conditions)?
- Were the project’s goals and approach still relevant throughout the implementation period?
Impact
Did the project contribute to the broader development goals or the improvement of the community?
- How did the project contribute to enhanced inter-community and intra-community acceptance?
- To what extent did the project activities impact the operations in the centers (if positively, how? And if negatively, how?):
- Catchment areas – project beneficiaries
- Self-reliance – accreditation
- Did the impact vary for different targeted areas (communities and centers)? If so, how, and why?
Sustainability
- What mechanisms or strategies were put in place to ensure the continuation of project benefits after the project ends?
- To what extent are the project outcomes likely to be maintained or scaled up in the future?
Coherence
- How well did the project align with other ongoing projects or initiatives in the area?
- How well was the project integrated within the broader organizational or sectoral strategy?
- What are the lessons learned, best practices, strengths and challenges in the project implementation?
Gender Impact assessment
- How did the project contribute to promoting gender equality and empowering women and marginalized groups?
- Were gender-specific objectives clearly defined and integrated into the project design and implementation?
- What measures were taken to ensure that gender equality outcomes are sustained after the project ends?
PCIA
- Did the project contribute to social cohesion or division within the community or target population?
- How did the project affect relationships between different social or political groups in the area?
Evaluation Methodology
Desk Study
- Review the project’s history, location, and implementation context.
- Review project documentation: proposal, dispensaries modality of operation, Memorandum of Understandings (MoUs), project reports
- Review publicly available similar project reports
Inception Phase
- Provide an inception report (including a description of the evaluation and gender impact assessment design and the sampling techniques and how data will be obtained and analyzed including a data collection planning worksheet or similar tool). The inception report shall furthermore contain an evaluation matrix specifying with which methods and along which criteria the evaluation questions will be answered, a tentative timeline as well as drafts of the reconstructed Theory of Change as well as the evaluation instruments (i.e. questionnaires, interview guides, etc.).
- The field phase will only take place upon the official approval of the inception report.
Field Phase
- Including involvement of project beneficiaries and other key stakeholders
- Data gathering and triangulation: it is expected that the evaluation will include quantitative and qualitative data. Various methods of data collection, such as analysis of documents, structured interviews, semi-structured interviews, face-to-face or by remote modalities, group discussions, surveys (online or in-person), and other methods shall be used. The application of innovative methodologies is encouraged. It is expected that the evaluation team adheres to ethical standards during data collection. A gender sensitive approach to data collection is to be applied and data shall be collected, analyzed and presented disaggregated by age, gender and if applicable, other vulnerability criteria. Throughout the entire evaluation process, the principles of do-no-harm shall be adhered to.
Data Review and Analysis
The document should include:
- Presentation of the Evaluation with Key Findings
- Presentation of the gender impact assessment
- Presentation of recommendations
- It is preferred that data is disaggregated by gender and age.
- Submission and presentation of the draft report to DKH and IOCC
- Period of comments and feedback from DKH and IOCC
- Submission of final report, inclusive of comments from all partners (DKH and IOCC) for final review.
- Submission of final report.
Evaluation Team Composition
To achieve the objectives of this assignment, the consultant or team should be an experienced individual or professional firm with the following skills:
- Demonstrates strong understanding of humanitarian and development issues, specifically related to Lebanon and the impact of the Syria crisis.
- Has experience working with European-donor-funded projects and evaluations.
- Evaluation Experience: Has conducted at least three evaluations in the past five years, ideally in relevant fields.
- Independence: Evaluation management should not have been involved in the design or implementation of the project being evaluated.
- Gender Focus: Experience with projects or assignments addressing the needs of women and girls, with knowledge of gender analysis and mainstreaming.
- Demonstrates cultural sensitivity in working with diverse communities.
- Willing and able to conduct work in the project targeted areas.
- Strong communication skills, fluent in both English and Arabic.
- Excellent English writing skills.
- For team/group applications, the team should be gender-balanced and diverse.
- Must uphold ethical standards, including impartiality and independence, in evaluation management.
Proposed Schedule & Deliverables
The consultant(s) is expected to complete the assignment within approximately 60 working days.
Work may begin on July 1, 2025 (with the evaluation kick-off meeting) and may conclude at the end of September 2025 (with the final report), depending on the agreed upon dates for the field work. An initial summative inception report (approximately 10 – 20 pages without annexes, including an evaluation matrix) should be submitted for discussion no later than July 21st, 2025. A precise timetable (by days) will be set by the evaluator(s) according to plans made and finalized with the inception report.
A draft of the final evaluation report [no less than 35 and no more than 60 pages, including a two-page executive summary with key findings and recommendations and excluding annexes and the results-assessment form (Annex 1)] will be submitted to IOCC and DKH no later than September 5, 2025. Findings and recommendations must be structured according to the evaluation questions. An outline of the final report structure should be agreed upon during the inception phase. At least the following must be completed: cover page); executive summary (should be included in the report and also as an independent document); background and objectives of the assignment; methodology (with mention of limitations, if any); summary of findings; analysis of findings addressing guiding questions; conclusions, lessons learned, and recommendations; and agenda of all activities carried out (annex).
The quality of the report will be judged according to the following criteria:
- Does the report contain a comprehensive and clear executive summary?
- Were the Terms of Reference fulfilled and is this reflected in the report?
- Is the report structured according to the OECD/DAC criteria?
- Are all evaluation questions answered?
- Are the methods and processes of the evaluation sufficiently documented in the evaluation report?
- Does the report describe and assess the intervention logic (e.g., log frame, program theory) and present/analyze a theory of change and its underlying assumptions?
- Are cross-cutting issues analyzed in the report?
- Are the conclusions and recommendations based on findings and are they clearly stated in the report?
- Does the report clearly differentiate between conclusions, lessons learned, and recommendations?
- Are the recommendations realistic and is it clearly expressed to whom the recommendations are addressed?
- Were the most significant stakeholders involved consulted?
- Does the report present the information contained in a presentable and clearly arranged form?
- Is the report free from spelling mistakes and unclear linguistic formulations?
- Can the report be distributed in the delivered form?
IOCC and DKH will review the report and submit comments within ten working days. The evaluator(s) will incorporate all comments and submit the final draft to IOCC no later than September 15, 2025, and the final report (including all annexes) no later than September 25, 2025. The evaluators will also need to present the findings no later than September 30, 2025.
All documents should be submitted in English. If an Arabic version (i.e., of survey questions) is provided, an accompanying version in English must be provided.
Roles and Responsibilities
DKH and IOCC will:
- Ensure that the consultant/firm have access to relevant information sources and documents related to the project.
- Validate the assessment questions and the data collection tools.
- Quality reviews of work delivered by the consultant/firm.
- Provide feedback on the methodology, tools, findings, conclusions, and recommendations for the assessment report.
- IOCC Program Tema will be responsible for managing the consultancy and coordination between stakeholders and the consultant/firm.
Ethical Standards
The consultant/firm should ensure the following ethical considerations while carrying out study activities:
- Informed Consent: Participants should be fully informed about the purpose, procedures, and use of research data. They must voluntarily agree to participate, and consent must be obtained without coercion.
- Confidentiality and Privacy: Personal information of participants must be protected. Data should be anonymized or de-identified, when possible, to ensure privacy, and participants should be aware of how their data will be used, stored, and shared.
- Minimizing Harm: Consultant/firm should minimize any potential physical, psychological, or social harm to participants.
- Integrity and Honesty: Honesty in data collection, reporting, and interpretation is crucial. Consultant/firm should avoid falsifying, fabricating, or misrepresenting data and ensure that results are presented transparently and accurately.
- Peer Review and Accountability: Consultant/firm should be subject to peer review and scrutiny to ensure validity and ethical adherence. Consultant/firm are accountable for their work and should be open to feedback and correction.
Proposal Assessment and Scoring
The contract shall be awarded to the most economical proposal, taking into account all the circumstances. The assessment shall be carried out on the basis of the criteria listed below and their respective weighting:
Award criterion Weighting of the award criterion in %
Quality of the technical offer 40%
Qualification of the evaluators 20%
Total price 40%
We reserve the right to award the contract already at the time of submission of the offer. We also reserve the right to conduct interviews with applicants in order to reach a decision.
Application instructions
Interested candidates are expected to submit narrative and financial proposals via sealed envelopes to IOCC offices. Offers should include:
Narrative:
- Cover letter summarizing experience in relation to this ToR
- The proposed methodology
- Planned and detailed schedule for the assignment
- CV (of all proposed parties)
- Writing sample from similar assignment
- Three professional references with contact information
Financial (in Euros):
- Detailed budget, including fees (with daily fees and expenses)
- Travel costs
- Any other anticipated costs.
Deadline for Offers:
Offers should be submitted to IOCC no later than June 6,2025, at 2:00 p.m. Beirut time to [email protected]
Only selected candidates will be contacted for an interview.
Please be sure to indicate you saw this position on Globaljobs.org
