Evaluation Quality Assessment Consultant (Home-Based)

Description

Job Title: Evaluation Quality Assessment Consultant (Home-Based)

Contract type: Individual consultancy

Duty Station: Home-based

 

Purpose of consultancy: 

The Independent Evaluation Office (IEO) seeks to recruit external consultants to serve as reviewers for the evaluation quality assessment (EQA) of UNFPA evaluations.

The UNFPA Evaluation Policy (2024) underscores the importance of high-quality evaluations to improve decision-making, accountability, and organizational learning. To ensure that evaluations meet quality standards, the IEO implements an Evaluation Quality Assurance and Assessment system. A key component of this system is the ex-post quality assessment, where evaluation reports are systematically assessed against the standards and criteria set in the EQA Grid (2025) by external independent reviewers.

The Evaluation Quality Assurance and Assessment system covers two main categories of evaluations:

• Centralized evaluations: These evaluations are commissioned by the IEO and typically assess issues of corporate strategic significance, development effectiveness, and organizational performance. They also include evaluations of selected large-scale, complex humanitarian responses at global, regional or country levels.
• Decentralized evaluations: These evaluations are commissioned by country offices, regional offices, and headquarters business units. They comprise country programme evaluations, regional programme evaluations, and project-level evaluations (evaluations of donor-funded projects and programmes), including humanitarian evaluations.

Aligned with the Evaluation Policy, the IEO, regional and country offices prepare multi-year costed evaluation plans which outline strategic evaluations to inform decision-making and programming. The plans are subject to periodic revision to reflect emerging priorities, organizational learning needs and special requests. All the evaluations included in the costed evaluation plans undergo internal quality assessment and external quality assessment.

The purpose of this consultancy is to provide independent, rigorous assessments of UNFPA centralized and decentralized evaluations to determine their overall quality, credibility, and utility for decision-making and organizational learning.

 

Scope of work:

The consultants will work under the overall oversight of the IEO Director and the direct supervision of the IEO EQA Focal Point. The consultants will be responsible for the following tasks:

1. Familiarization with UNFPA evaluation guidance and EQA framework:

• Review the UNFPA Guidance on Evaluation Quality Assurance and Assessment (2024) and the annotated EQA grid (2025) providing practical guidance for rating the quality criteria.
• Review the UNFPA Evaluation Handbook (2024), Country Programme Evaluation Management Kit (2025), the Compendium to the UNFPA Evaluation Handbook on Humanitarian Evaluation (2024) and other guidelines of UNFPA and the United Nations Evaluation Group (UNEG) on integrating cross-cutting issues into evaluation, including human rights, gender equality and disability inclusion.
• Participate in an orientation on the EQA grid to ensure a consistent understanding of the rating criteria and quality standards, as well as periodic refresher training to maintain consistent EQA practice.

2. Assessment of UNFPA evaluation reports:

• Review assigned centralized and decentralized evaluation reports (including project-level evaluations and humanitarian evaluations) and supporting documentation (final evaluation report annexes, including terms of reference, design report and annexes, case studies, etc.) in English, French, Portuguese, or Spanish.
• Assess the evaluation reports against the EQA grid, referring to the annotated version for practical guidance on how to interpret and rate the criteria set in the grid correctly.
• Assign a rating for each indicator in the grid, provide detailed justification for the rating, synthesize key results into a narrative that highlights strengths and weaknesses, and provide specific recommendations for future quality improvement.
• Provide ratings on the UN SWAP Evaluation Performance Indicator as guided by the UN SWAP 3.0 Framework and Technical Guidance.
• Provide ratings on disability inclusion as per the Guidance on Integrating Disability Inclusion in Evaluations and Reporting on the UNDIS Entity Accountability Framework Evaluation Indicator 

3. Feedback to improve UNFPA EQA framework

• Contribute to the continuous refinement of the EQA framework by providing feedback on the EQA grid and annotations, identifying areas that require further clarification and suggesting improvements to enhance consistency in ratings across reviewers.
• Participate in annual review meetings with the IEO to discuss EQA performance, including improvements to the EQA framework and potential actions to address quality gaps and weaknesses in UNFPA evaluations.

 

The IEO will provide access to the evaluation reports to be assessed, relevant supporting documents (terms of reference, design reports, case studies, etc.), the guidance for Evaluation Quality Assurance and Assessment and the EQA grid. In addition, it will share guidance for evaluation at UNFPA, including decentralized and humanitarian evaluations.

 

Deliverables:

The consultants shall submit the following deliverables:

• A completed EQA grid for each assigned evaluation, submitted in the corresponding language of the report (English, French, Portuguese, or Spanish). This includes a narrative summary that outlines key strengths and weaknesses and offers recommendations for future quality improvement.

Assessments are expected to be returned to the IEO within 10 working days following submission for review. All deliverables must be submitted electronically.

All deliverables are subject to IEO approval. The consultants shall remain available to provide clarifications and make necessary revisions to ensure accuracy and consistency across assessments. Payment will be processed upon the IEO’s certification of satisfactory performance. Remuneration is based on a fixed unit rate per completed quality assessment, differentiated by type of evaluation. The total payment will be calculated as a lump sum based on the number of approved assessments completed for each type of evaluation at the respective agreed rates.

The estimated level of effort is determined by the type of evaluation being assessed. 

• Centralized evaluations and specific decentralized evaluations (regional and country programme evaluations): 1.5 working days per quality assessment.
• Other decentralized evaluations (project-level and humanitarian evaluations): 1.0 working days per quality assessment.

Assignments are typically concentrated in the first and fourth quarters of the calendar year, aligning with UNFPA’s country programme evaluation cycle. Deadlines and estimated levels of effort will be communicated upon assignment of each evaluation report.

The consultants are responsible for providing and utilizing all equipment required to complete assignments, including a computer, suitable communication technology, and a secure internet connection.

 

Time period:

The work will be intermittent and upon request. The number of evaluation reports assigned to each consultant will vary annually, depending on the volume of centralized and decentralized evaluations commissioned by UNFPA, as well as the consultant’s availability, language skills, and performance. The IEO does not commit to a minimum number of assignments.

The IEO will share annual plans for centralized and decentralized evaluations and provide advance notice of upcoming needs for EQA, where possible. For an indication of the number of evaluations to be assessed, please refer to the UNFPA Multi-Year Costed Evaluation Plan, 2024-2027, and the costed evaluation plans of UNFPA regional and country offices.

The agreement will be valid for a period of 11 months, with the possibility of extension based on performance and needs of the IEO.

 

Expected travel: 

The consultancy is home-based. No travel is required.

 

Qualifications and experience:

Education

• Advanced university degree (Master’s degree or equivalent) in evaluation, public health, demography, social sciences, international development, or a related field.

 

Knowledge and experience

• A minimum of seven years of professional experience in evaluation, preferably with UNFPA or other United Nations agencies in both development and humanitarian contexts.
• Experience in conducting evaluation quality assurance, assessment or meta-evaluation is desirable.
• Familiarity with evaluation quality standards, including UNEG norms and standards, OECD/DAC evaluation criteria, ALNAP evaluation criteria, and IASC guidelines.
• Extensive knowledge of diverse evaluation approaches (theory-based, mixed-methods, utilization-focused, outcome harvesting), sampling strategies (probability and non-probability), data collection and analysis methods (qualitative and quantitative), and data triangulation.
• Strong expertise in sexual and reproductive health, gender equality and empowerment of women, adolescents and youth, and/or population dynamics and data, including in humanitarian contexts.
• Experience in integrating human rights, gender equality and disability inclusion in evaluation.
• Strong analytical skills, with the ability to synthesize large volumes of information.
• Excellent written communication skills.
• Ability to manage multiple assignments and meet deadlines.

 

Languages

• Fluency in English (reading and writing) is required.
• Working proficiency in French, Portuguese, and/or Spanish is required for assessments of evaluation reports in those languages.
• Fluency in English and one or more working language is an advantage.

 

Other information:

Conflict of interest and independence

To ensure independence, the consultants must not have had any prior involvement with the intervention evaluated nor with the evaluation team that conducted the report being reviewed

 

Ethical and responsible use of artificial intelligence (AI) in evaluation work 

AI technologies cannot be used in the framework of this contract unless a prior written agreement is obtained from the IEO. Upon this prior agreement, the consultants are obligated to disclose the utilization of AI tools in evaluation and commits to upholding ethical standards and accuracy in the application of AI tools.

1. Prior approval for utilization of AI tools: The use of AI tools must be explicitly agreed upon and approved in writing by the IEO Director.
2. Declaration of the utilization of AI tools: If the use of AI tools in evaluation is agreed upon with the IEO, the consultants must be transparent and declare the use of AI tools in evaluation work and other work-related tasks, specifying the nature of AI usage. The AI tools utilized in work-related tasks must include only those tools that are vetted by the IEO.
3. Verification of accuracy: The consultants commit to diligently checking the accuracy of AI-generated results and assumes full responsibility for its reliability and validity.
4. Ethical and responsible use: The consultants are obligated to uphold ethical principles in the use of AI in work-related tasks, as well as relevant regulations that govern the use of AI in the United Nations system. This includes the Digital & Technology Network Guidance on the Use of Generative AI Tools in the United Nations System, Principles for the Ethical Use of Artificial Intelligence in the United Nations System, and UNFPA Information Security Policy. The consultants commit to employing AI tools that adhere to principles of non-discrimination, fairness, transparency, and accountability. The consultants will adopt an approach that aligns with the principle of ‘leaving no one behind’, ensuring that AI tool usage avoids exclusion or disadvantage to any group.

 

How to Apply:

Beyond completing the application form, including the sections on education, employment history and languages, applicants must attach the following documents in Quantum:

• A cover letter, indicating the candidate’s motivation for this consultancy as well as relevant expertise and experience.
• A financial proposal, specifying the proposed unit rate for: (i) a quality assessment of centralized and specific decentralized evaluations; and (ii) other decentralized evaluations.

  Only those candidates who meet all qualifications and experience will be contacted for further consideration. Incomplete applications will be automatically disqualified.

 

Required Competencies:

Values:

• Exemplifying integrity, 
• Demonstrating commitment to UNFPA and the UN system, 
• Embracing cultural diversity, 
• Embracing change

Core Competencies: 

• Achieving results,
• Being accountable,
• Developing and applying professional expertise/business acumen,
• Thinking analytically and strategically,
• Working in teams/managing ourselves and our relationships,