Annex 4: Monitoring and Evaluation tools

Instructions: This document provides an overview of the different common data collection methods and shows both the advantages and limitations of each. Before conducting an evaluation, a close review of the document is recommended to consider the type and specific framework for the evaluation.

Method Description Advantages Limitations

Desk review of existing reports and documents

Existing documentation, including quantitative and qualitative information about the project and its outputs and outcomes, such as documentation from capacity development activities, donor reports, digital records and other evidence.

The information exists and is accessible at a low cost.

May be time-consuming to put together and analyse.

Evidence can be difficult to codify and analyse.

Difficult to verify reliability and validity of data.

Questionnaires

Provide a standardized approach to obtaining information on a wide range of topics from a large number or diversity of stakeholders (usually employing sampling techniques) to obtain information on their attitudes, beliefs, opinions, perceptions, level of satisfaction, and so forth, concerning the operations, inputs, outputs and contextual factors of a project.

Good for quickly gathering descriptive data on a wide range of topics at a relatively low cost.

May be easier to analyse.

May lead to bias, such as social desirability bias.

May provide a general picture but may lack depth.

May provide information out of context.

Data may be subject to sampling bias.

For online surveys, the number of respondents may not be controlled.

In-depth interviews

Solicit person-to-person responses to questions designed to obtain in-depth information about a person’s impressions or experiences, or to learn more about his or her answers to questionnaires or surveys.

Facilitates fuller coverage, range and depth of information on a topic.

Can be difficult to analyse.

Potential for interviewer to bias against participant’s responses.

Method Description Advantages Limitations

On-site observation

Entails use of observation form to record accurate information on site about how a project operates.

Can see operations of a project as they occur.

Can adapt to events as they occur.

Can be difficult to interpret observed behaviours.

Subject to site selection bias.

Focus groups

Small group (6 to 12 people) discussion to explore stakeholder opinions and judgements towards an activity, process, project or policy. They can also be used to collect in-depth information on the needs, motivations, intentions and experiences of the group.

Useful to obtain in-depth qualitative information.

Requires qualified facilitator.

Can be difficult to analyse and interpret.

Subject to facilitator bias.

Key informants

Qualitative in-depth interviews, often one-onone, with a wide range of stakeholders who have firsthand knowledge about the initiative’s operations and context. These community experts can provide particular knowledge and understanding of problems and recommend solutions.

Can provide insight on the nature of problems and recommend solutions.

Can provide different perspectives on a single issue or on several issues.

Subject to sampling bias.

Must have some means to verify or corroborate information.

Case studies

Involves comprehensive examination of cases to obtain in-depth information with the goal to fully understand the operational dynamics, activities, outputs, outcomes and interactions of a development project.

Useful to fully explore factors that contribute to outputs and outcomes.

Requires considerable time and resources not usually available for commissioned evaluations.

Can be difficult to analyse and not necessarily replicable.

This form is designed to determine to what extent the reintegration process of a migrant has been sustainable, that is to what extent a condition has been achieved “where returnees have reached a level of economic self-sufficiency, social stability within their community and psychosocial well-being that enables them to cope with (re)migration drivers”.97 Aiming to cover all aspects of this definition of sustainability, the questions below cover the three different dimensions of reintegration: economic, social and psychosocial. Without prejudice to the importance of the assistance provided by IOM through AVRR/PARA programmes, the main focus is not to assess the satisfaction of the migrant with IOM’s assistance throughout the reintegration process, but to evaluate to what extent the migrant’s reintegration has been sustainable.

The survey, which has been kept as short as possible, enables IOM staff to generate a composite (overall) reintegration score, as well as separate scores for the sustainability of reintegration in the economic, social and psychosocial dimensions. The example below of three respondents from IOM’s MEASURE project illustrates how reintegration experiences vary widely. The scoring system presents an opportunity to understand individual reintegration needs with a new level of insight.

Annex 4.1

On an individual level, these scores can be easily visualized similarly to the displays above. These images show the programmatic value of having individual dimensional scores: for example, while the 44-year old Ethiopian returnee needs significant assistance across all dimensions, the scores show that he is particularly vulnerable in the economic dimension. Similarly, while the 19-year old Afghan returnee is very well reintegrated overall, interventions should primarily focus on his psychosocial needs. Finally, the 24-year old Iraqi returnee appears to be better reintegrated in the social and psychosocial dimensions but requires a more intensive approach to her economic reintegration.

The scoring system, as well as interpretation of resulting scores, is further explained in a methodological note on scoring reintegration sustainability. For a copy of the methodological note, please contact: MPAHQTeam@iom.int. The methodological note also offers further guidance on the use of the survey, such as timing. IOM staff are advised to familiarize themselves with the methodological note before proceeding to study the survey itself as outlined below.

This form should be completed by staff during a structured interview with the returnee. The survey can serve as a baseline and progress assessment before and during the period of reintegration assistance and for final evaluation of returnee sustainability after the provision of reintegration assistance was concluded, as outlined below:

Annex 4.2

It is recommended that this survey is administered in person by a staff member who is or was not directly responsible for the provision of reintegration assistance to the beneficiary. This increases the likelihood that respondents will express their feelings without hesitation, avoiding what is termed “social desirability bias.”

This document offers a closer look at the indicators and questions used, guiding staff through the exact interpretation of each indicator. Page 2 contains the survey protocol, Pages 3–9 contain the survey template, annotated to facilitate understanding of the indicators.

SURVEY PROTOCOL

The survey should be conducted in a private space where returnees may feel comfortable reflecting on their experience and answering potentially sensitive questions. They should never be forced to answer any question and have the right to interrupt the interview at any time.

Protocol:

  1. Prior to meeting, the staff member completes Profile and Section 1 of the survey. Information should be verified with the beneficiary, and any outstanding questions from these sections answered. Categories “selected” in Section 1 determine the composition of the survey questionnaire later administered to each returnee. (Staff only ask questions in sections “selected” in Section 1.)

  2. Interviewer reads prompt to beneficiary and seeks their consent.98 If obtained, interviewer proceeds to Section 2.

  3. For all questions: Interviewer reads questions out loud.

  4. Interviewer observes instructions below each question:

“prompt” indicates that the interviewer should read answer options, and allow respondent to select the most appropriate.

“do not prompt” indicates that the interviewer should not read a list of possible answers to the respondent. Instead, interviewer should listen to the respondent’s free response, and select answer(s) closest to their own words.

“select one” indicates that the question can only have one answer.

“select all applicable” indicates that the question can have multiple answers.

  1. Interviewer records answers and notes.

  2. If selected answers refer to follow-up questions (such as in Question no. 9), Interviewer proceeds to follow up question (marked by question number in brackets, for instance “(10)”).

Annex 4.3

REINTEGRATION SUSTAINABILITY SURVEY

Annex 4.4

Reintegration sustainability

ECONOMIC DIMENSION Questions 1–10 contain indicators of economic reintegration, which contribute to economic self-sufficiency

  Questions Answers Notes
1

How satisfied are you with your current economic situation?
(Overall economic situation, self-assessed by respondent

select one
do not prompt

  • Very satisfied
  • Satisfied
  • OK
  • Dissatisfied ➔ please explain
  • Very Dissatisfied ➔ please explain
  • I don't wish to answer

For staff needs, and follow-up explanations. Anything in this column is not used for score calculation, but could be useful for case management purposes.

2

Since you returned, how often have you had to reduce the quantity or quality of food you eat because of its cost?
(Food rationing as a cost-reduction strategy is a strong indicator of an unstable economic situation.)

select one
do not prompt

  • Very often
  • Often
  • Sometimes
  • Rarely
  • Never
  • I don't wish to answer

Given that this indicator is cross-sectional (has implications also for social and psychosocial dimensions of reintegration), it is weighted more heavily in the scoring system to reflect its overall importance in determining sustainability of reintegration. More information is available in the methodological note.

3

Are you able to borrow money if you need to?
(Perceived availability of credit, regardless of source – bank, family, friends, traditional loans system, microcredit – and regardless of whether respondent is effectively taking out loans or not.)

select one
do not prompt

  • Yes
  • No
  • I don’t know
  • I don’t wish to answer
 
4

Do you borrow money? How frequently?
(Behaviour self-reported by respondent, regardless of source of credit and amount – even very small amounts count)

select one
do not prompt

  • Very often
  • Often
  • OK
  • Sometimes
  • Rarely
  • Never (I don’t borrow money)
  • I don't wish to answer
 
  Questions Answers Notes
5

On average, which amount is bigger: your spending every month, or your debt?
(The comparison allows us to see whether respondent is able to cover their monthly expenses from earnings, or supplements basic life needs with loans, a much less sustainable behaviour.)

select one
do not prompt

  • I don’t have debt
  • Debt is larger
  • OK
  • Spending is larger
  • I don't wish to answer
  • N/A
 
6

How would you rate your access to opportunities (employment and training)?
(Perceived, personal ability to reach and get opportunities for income generation – jobs, courses for skills enhancement and so on.)

select one
do not prompt

  • Very good
  • Good
  • Fair
  • Poor
  • Very poor
  • I don't know
 
7

Do you currently work?
(Either employment or selfemployment, formal or informal. If respondent currently in unpaid training or attending school, select “N/A”.)

select one
do not prompt

  • Yes
  • No
  • I don’t wish to answer
  • N/A
 
8

Do you own any of the following productive assets?
(Productive assets create a potential basis for an income-generating activity. As categories will differ based on context, it is suggested that interviewers consider the potential of assets in local economies and adapt answers accordingly. For scoring purposes, it is only necessary to know if respondent does (yes) or does not (no) own a productive asset of any kind. However, knowing which particular asset a returnee owns will support the case for management and reintegration counselling.)

select all applicable
prompt

  • Land
  • Animals
  • Trees (fruits, nuts)
  • Buildings and Structures
  • Rarely
  • Vehicles
  • Equipment and Tools
  • Other - please explain ➔...
  • I don't wish to answer
 
  Questions Answers Notes
9

Are you currently looking for a job?
(Regardless of currently working or not. A respondent might be employed but unhappy with their current pay and conditions and so forth, and searching for alternative opportunities.)

select one
do not prompt

  • Yes (please continue to Q10)
  • No (please continue to Q11)
  • I don’t wish to answer (Q11)

If respondent indicates YES as an answer, please do include Q10. If respondent indicates NO or I DON’T WISH TO ANSWER, please skip Q10, and continue to Q11.

10

Why are you looking for a new job?

only if “yes” selected aboveselect all applicable
prompt

  • Unemployed
  • Unhappy with work at current job
  • Unhappy with work conditions (location, working hours and so on)
  • Unhappy with salary at current job
  • Other - please explain ➔……
 

SOCIAL DIMENSION Questions 11–21 contain indicators of social reintegration, reflecting the extent to which returnees have reached social stability within their community, including access to services relating to housing, education, justice, health and other public infrastructure services.

11

How would you rate your access to housing in your community?
(Self-assessed ability to find, change and afford housing)

select one
do not prompt

  • Very good
  • Good
  • Fair
  • Poor
  • Very poor
  • I don’t know
  • I don’t wish to answer
 
12

How would you rate the standard of housing you live in today?
(Self-assessment of standard of housing – safety, cleanliness, size, neighbourhood and other conditions.)

select one
prompt if needed

  • Very good
  • Good
  • Average
  • Poor
  • Very poor
  • I don’t wish to answer
 
  Questions Answers Notes
13

How would you rate the access to education in your community?
(Self-assessed ability to take part in educational activities, programmes, courses, and so on)

select one
do not prompt

  • Very good
  • Good
  • Fair
  • Poor
  • Very poor
  • I don’t know
 
14

Are all school-aged children in your household currently attending school?
(This includes children to whom respondent is a parent or guardian, as well as other children in respondents’ household.)

select onedo not prompt

  • Yes (also select if no children in home)
  • No - some but not all ➔ please explain
  • None ➔ please explain
  • I don’t wish to answer
 
15

How would you rate the access to justice and law enforcement in your community?
(Self-assessed ability to use and be protected by services and guarantees provided by courts, police, military, and so on.)

select one
do not prompt

  • Very good
  • Good
  • Fair
  • Poor
  • Very poor
  • I don’t know
  • I don’t wish to answer
 
16

Do you have at least one identification document?
(Passport, national or local identification document, birth certificate – adjust specifics based on local context.)

select one
do not prompt

  • Yes
  • No
  • I don’t know
  • I don’t wish to answer
 
17

How would you rate the access to documentation (personal ID, birth certificates and so on) in your community?
(Self-assessed ability to request and receive personal documents issued by the State.)

select one
do not prompt

  • Very good
  • Good
  • Fair
  • Poor
  • Very poor
  • I don’t know
 
  Questions Answers Notes
18

How would you rate the access to safe drinking water in your community?
(Self-assessed ability to access and use water which is suitable for drinking and hygiene.)

select one
do not prompt

  • Very good
  • Good
  • Fair
  • Poor
  • Very poor
  • I don’t know
  • I don’t wish to answer
 
19

How would you rate the access to health care in your community?
(Self-assessed ability to access and use medical services)

select onedo not prompt

  • Very good
  • Good
  • Fair
  • Poor ➔ ..... please explain
  • Very poor ➔ ..... please explain
  • I don’t wish to answer

Please explain why health care is not easily accessible to you:

  • No health-care facility exists nearby
  • It is too expensive
  • It is too far
  • Other:

 

20

What is the quality of health care available to you?
(Self-perceived standard of care, which respondent is able to obtain for themselves.)

select one
prompt if needed

  • Very good
  • Good
  • Fair
  • Bad
  • Very bad
  • I don’t know
  • I don’t wish to answer
 
21

Access to public services overall is generated from average answers to above questions (Q13, 15, 17, 18, 19).

PSYCHOSOCIAL DIMENSION Questions 22–32 contain indicators of psychosocial reintegration, encompassing the emotional and psychological elements of reintegration.

22

How often are you invited or do you participate in social activities (celebrations, weddings, other events) within your community?
(Both invitations and participation matter, showing strength of personal connections to community.

select one
do not prompt

  • Very often
  • Often
  • Sometimes
  • Rarely
  • Never
  • I don’t wish to answer
 
  Questions Answers Notes
23

How do you feel about your support network? Can you rely on the network’s support?
(Self-perceived support network which can provide emotional or practical help in time of need, regardless of actual type, size, strength of support.)

select one
do not prompt

  • Very goodd - a very strong network
  • Good
  • Fair
  • Bad
  • Very bad - a very weak network
  • I don’t know
  • I don’t wish to answer
 
24

Do you feel you are part of the community where you currently live?
(Personal feeling of belonging.)

select one
do not prompt

  • I agree - I feel strongly that I am part of the community
  • I somewhat agree
  • I don’t agree or disagree
  • I somewhat disagree
  • I strongly disagree - I don’t feel part of the community at all
  • I don’t know
  • I don’t wish to answer
 
25

How physically safe do you feel for yourself and your family during everyday activities outside?
(Perceived physical safety from violence and persecution and other forms of insecurity. May be related to belonging to a social group or to the status of returnee alone.)

select one
do not prompt

  • I feel very safe all the time
  • I feel safe most of the time
  • Neutral
  • I feel unsafe most of the time
  • I feel very unsafe all the time
  • I don’t wish to answer

Given that this indicator is cross-sectional (has implications also for social and economic dimensions of reintegration), it is weighted more heavily in the scoring system to reflect its overall importance in determining sustainability of reintegration.

26

How frequently have you experienced important tensions or conflicts between you and your family since you returned?
(Self-perceived frequency. Every family experiences or is accustomed to a different frequency of conflicts – this question asks about conflicts and tensions that feel subjectively important and disturbing to the returnee, therefore hampering the reintegration process. These tensions could be new or dating prior to return.)

select one
do not prompt

  • Very often
  • Often
  • Sometimes
  • Rarely
  • Never
  • I don’t wish to answer

For case management follow up: do you experience more incidents of tension than before your migration experience?

  Questions Answers Notes
27

Have you felt discriminated against since your return?
(Frequency of a feeling, no need for additional information on specific instances of discrimination.)
Definition: discrimination entails inability to enjoy rights and freedoms without distinction of any kind, such as race, colour, sex, language, religion, political or other opinion, national or social origin, property, birth or other status.

select one
do not prompt

  • Never
  • Only rarely
  • Sometimes ➔ …… please explain
  • Very often ➔ …… please explain
  • I don’t wish to answer

Follow up: if yes, please explain.

28

Do you often suffer from any of the following?
- Feeling angry
- Feeling sad
- Feeling afraid
- Feeling stressed
- Feeling lonely
- Feeling low self-worth
- Difficulty concentrating
(Signs of psychosocial distress, answer should consider frequency of these symptoms.)

prompt
select one

  • Never
  • Only rarely
  • Sometimes ➔ …… please explain
  • Very often ➔ …… please explain
  • I don’t wish to answer
 
29

Would you wish to receive specialized psychological support?
(Such support may include informal or formal counselling, and other forms of support. Does not refer exclusively to psychological therapy.)

select one
do not prompt

  • Yes
  • No
  • I don’t know
  • I don’t wish to answer
 
30

Do you feel that you are able to stay and live in this country?
(Focus on ability to stay in country of origin, as opposed to wish, is given by IOM’s definition of sustainable reintegration: “Having achieved sustainable reintegration, returnees are able to make further migration decisions a matter of choice, rather than necessity.”)

select one
do not prompt

  • Yes
  • No (please continue to Q32)
  • I don’t know
  • I don’t wish to answer

Given that this indicator is cross-sectional (has implications also for social and economic dimensions of reintegration), it is weighted more heavily in the scoring system to reflect its overall importance in determining sustainability of reintegration.

  Questions Answers Notes
31

What is it that makes you feel that way?
(Important distinction between the need and the wish to leave – reflecting the respondent’s ability to deal with remigration drivers in country of origin. If respondent indicates both wish and need to leave, please select primary reason. For example, if a respondent has been struggling to find employment, is unable to cover their basic needs, and also misses their girlfriend in Belgium select “need” – since inability to establish sustainable living is the primary underlining reason for wanting to leave.)

only if “no” answered above
select one
do not prompt

  • I miss my friends/family members elsewhere; cultural factors; wish to continue studies abroad (WISH TO LEAVE)
  • Lack of jobs; lack of security; low earnings; lack of essential services; family pressure (FEEL THE NEED TO LEAVE)
 
32

Who are the people and organizations that support you in this community?

select all applicable
do not prompt initially

  • Family
  • Friends
  • Religious organizations and leaders
  • Community leaders
  • Work colleagues
  • IOM
  • NGOs
  • Other returnees
  • Other - please explain ➔ ……
  • No one
 

97 For IOM definition of sustainable reintegration, see Towards an Integrated Approach to Reintegration in the Context of Return (IOM, 2017).

98 When conducted in person, consent should be written. When interview conducted by phone, explicit, beneficiary should be asked to give explicit, informed consent verbally.

99  This survey can be taken repeatedly to show progress in reintegration sustainability following migrants’ return. Please refer to methodological note for further information.

TITLE [MIDTERM/FINAL/OTHER EVALUATION FOR “PROJECT”]

Commissioned by: Specify who is commissioning the evaluation report.

Evaluation context

Write a few paragraphs about the context of the evaluation. A few paragraphs about the project(s) that is to be evaluated and a general description of the relevant political, environmental, social, economic and legal context is usually sufficient.

Evaluation purpose

In this section, briefly explain why the evaluation is being conducted and why it is being conducted at this time. Specify the intended audience for the evaluation and how the evaluation will be used.

Some examples of audience and purpose are as follows:

  • The evaluation is being conducted for use by management, so that they can improve the implementation of an ongoing set of activities, projects or programmes.

  • The evaluation is being conducted for use by stakeholders, so that they can assess the relevance and accountability of a project for intended beneficiaries.

  • The evaluation is being conducted for use by the project team, so that they can document lessons learned and best practices from a completed set of activities.

  • The evaluation is being conducted for use by a donor, so that they can assess value for money for a set of activities that they have funded.

  • The evaluation is being conducted for use by senior management, so they can assess organizational effectiveness in implementing a strategy.

It is fairly common for an evaluation to be intended for use by a variety of audiences, such as project management, senior management, stakeholders and donors. If this is the case, briefly describe all of the evaluation’s main intended audiences and uses. Keep in mind the principle of intentionality in evaluations, which means that evaluations should only be undertaken if there is a clear intention to use the evaluation findings (refer to UNEG Norms for Evaluation in the UN system).

Evaluation scope

Briefly describe what the evaluation will cover and will not cover. This should include the time period to be covered (that is, the intervention period being evaluated, not the period of time available to complete the evaluation), the phases of a project to be covered and the geographical area to be covered. If there is a specific project, state its name. If there are specific exclusions – for example, if a project is being implemented in six provinces but two are inaccessible and will not be included in the evaluation – state them clearly.

Make sure that the evaluation scope is sufficient to achieve the evaluation purpose. For example, if the purpose is to assess value for money, but only the first three months of project implementation are being evaluated, the evaluation is unlikely to be able to achieve its purpose. Similarly, ensure that the scope of the evaluation is feasible within time and resource restraints.

Evaluation criteria

Specifically list the evaluation criteria that will form the basis of the evaluation.

Evaluation questions

For each of the listed criteria, specify the evaluation questions that the evaluator will answer. Cluster them according to the criteria. These questions should be specifically tailored to the needs of this evaluation.

Evaluation methodology

In this section, describe the data collection and analysis methods that will be used to conduct the evaluation. Refer to Annex 4.A for a description of different data collection and analysis methods. Indicate how the evaluation will address relevant cross-cutting themes of the rights-based approach to programming, gender mainstreaming, environmental sensitivity and sustainability, sustainability of results, principled humanitarian action and mainstreaming protection into crisis response.

Bear in mind that it might be necessary for this section to be more general in nature, pending development of a more detailed methodology following discussions with the selected evaluator or evaluation team. This is particularly the case when the evaluation manager lacks technical expertise and intends to solicit the advice of the evaluator on the most appropriate methodologies for the evaluation.

Finally, state that the evaluation must follow UNEG norms and standards for evaluations, and relevant ethical guidelines.

Evaluation deliverables

List the deliverables the evaluator will be responsible for providing. This usually includes an inception report, a presentation outlining the initial findings and a final report.

Evaluation workplan

In this section, describe the following:

  • The activities to be conducted and the amount of time (how many days, weeks or months) that will be allocated for completing each activity.

  • The roles and responsibilities of each member of the evaluation team and of the stakeholders.

  • The processes for quality assurance. At a minimum, this should include: (a) the agreement on the final terms of reference between the evaluation manager and the evaluator or evaluation team; (b) review, revision and acceptance of the inception report; (c) review, revision and acceptance of the final report. It is also standard practice to have a management meeting at the beginning of the evaluation process to ensure that the evaluation manager, the evaluator or evaluation team, and stakeholders (if relevant) all share a common understanding of the evaluation process and various roles and responsibilities, as well as to have a debrief and presentation of initial findings following conclusion of the data collection and preliminary analysis. This allows for any obvious oversights, misinterpretations or information gaps to be identified and addressed before the evaluator begins drafting the final report.

This information can be provided either in narrative text or in the table below. If using both the narrative text and the table, review the information carefully to ensure that what is written in the narrative matches with what is written in the table.

Activity Days Responsible Location Days/Weeks/Months
1 2 3 4 5 6
Insert individual activities to be conducted during the evaluation. Indicate how many days are needed for each activity. Specify who is responsible for completing the activity Specify where the activity is to be conducted.            
Example: Review project documents and relevant literature. 3 Evaluator Home based            
                   
                   
                   

Evaluation budget

Inclusion of this section is at the discretion of the evaluation manager. In some contexts, it is appropriate to simply specify the total budget available for the evaluation or to provide a more detailed budget (such as the amount to be paid upon receipt and acceptance of each deliverable or to specify the amount available for fees, travel, daily subsistence allowance, equipment, data collection and others). In other contexts, it may be preferable to not specify the budget and instead have applicants propose a budget in their applications.

Instructions: This checklist provides guidance on the different steps to be undertaken during an evaluation and at what stage; confirmation that no crucial step has been forgotten is vital for the evaluation.

Preparation of the Evaluation

The overall objective and purpose of the evaluation has been defined (analysis of AVRR programme’s performance and accountability, exploration of new modalities for implementation and so on).

 

The focus and scope of the evaluation has been defined (focus is mainly related to evaluation criteria of relevance, effectiveness, cost-benefit, efficiency, outcome, sustainability and long-term impact of the AVRR programme).

 

A decision has been taken as to whether the evaluation will be carried out by an internal or external evaluator and budget provision guaranteed.

 

The methods for data collection have been defined (review of existing documents and report, questionnaires, in-depth interviews, on-site evaluation, focus groups, key informants and case studies) in line with the timing and resources available for the evaluation.

 

The Terms of Reference for the evaluation have been drafted, having considered the following elements below:

- The Background section summarizes the context of the project that will be evaluated. The expected outcomes and outputs of the projects are stated as they will be one of the main references of the evaluation (to list indicators could be too detailed unless only a few indicators were listed in the initial project document).

- The objective(s) of the evaluation specify the ‘why do it’, the nature of the evaluation to be undertaken and the product it is meant to deliver, the intended audience, the use of the evaluation and the involvement of the stakeholders in the evaluation.

- The methodology section covers the approach for data collection and data analyses in a precise manner, ensuring that the choice for the duration as well as the techniques to be applied during the evaluation adequately reflect the available budget (taking into account potentially high costs in the event that a large number of interviews are carried out with returnees in different countries of origin).

- The role of the various parties involved in the evaluation (IOM, project partners, beneficiaries and, if included, steering committees) is clearly defined, enabling all parties to know what they are responsible for and what is expected from them, such as providing information on the management of the project, allowing access to project–related documentation and collecting data from the government.

- The budget lays out (if possible, in detail) the resources required to conduct the evaluation, including potential consultancy fees and costs of data collection and surveys; the resources in kind (such as transportation or administrative support) which will be made available for the evaluation team, are clearly reflected.

- The team composition is described (a single evaluator or a team with different expertise and skills).

- Deliverables that will be generated at various stages of the evaluation process (such as work plan, inception report, mid-term report, final report and recommendations) are included.

 

- The schedule sets out in chronological order the dates by which certain activities have to be completed. This includes a consideration of possible risks that might have an impact on the timing of the evaluation (such as being unable to contact migrants for monitoring purposes upon return).

- Relevant cross-cutting aspects, such as gender and human rights are duly considered in the ToRs and in the evaluation as a whole.

- Data protection principles are embedded in the evaluation’s methodology.

- An ethical framework is established for the inclusion of vulnerable groups.

- Adherence to UNEG Norms, Standards and Ethical Guidelines for Evaluation and Evaluators.100

 

 

Managing and Implementing Evaluation

The evaluation consultant or team has been selected, based on the following considerations:

- The evaluator(s) has the appropriate educational background and training for the evaluation (social sciences, specialized training in evaluation, project management, social statistics or statistical research and analysis, specific expertise such as economics or microcredits, all depending on the nature of the evaluation).

- The evaluator(s) has sufficient background and experience with AVRR or IOM or UN evaluations in general, and with the different methodologies identified for data collection in particular,

- The evaluator(s) has sufficient knowledge about the other areas to be evaluated (for instance AVRR polices and legislation) as well as of the local context (host vs. origin country, social and economic situations, security and stabilization policies) in which the evaluation is taking place.

- The evaluator(s) has appropriate oral and written communication skills.

- If the evaluator(s) is given access to confidential information, a confidentiality agreement has been signed with them.

 

A preparatory workshop has been carried out to discuss relevant aspects of the evaluation, such as clarifying the roles and coordination of the various stakeholders (in particular when adopting participatory approaches).

 

The project evaluator(s) has been introduced to the AVRR project team and other relevant stakeholders, and is briefed about the nature and objective of the evaluation.

 

Assistance to the evaluator(s) is provided by the AVRR project team throughout the process of data collection as needed (such as by arranging interviews with migrants and other actors, identifying respondents for questionnaires, organizing site visits to returnees’ places of work or meetings).

 

Follow-up meetings are organized between the reintegration team and the evaluator(s) to monitor the work and provide input, if needed, respecting the independence of the evaluator

 

If foreseen in the ToRs, the inception report and/or an interim report is shared with the AVRR project team or Chief of Mission or relevant stakeholders for their inputs.

 

The final evaluation report responds clearly to the objectives of the evaluation, is logically structured and contains evidence-based findings, conclusions, recommendations and lessons.

 

The project team is given the opportunity to provide input with regards to the content, structure, and length of the report, keeping in mind the independence of the evaluators regarding the actual findings and recommendations of the report.

 

A quality review of the final evaluation report is conducted prior to publication, including a revision as to whether the report addresses the objectives of the evaluation, that it has been well prepared and is clearly presented.101

 

A review of the findings and recommendations of the final report takes place in coordination with relevant stakeholders.

 

A debriefing (such as a workshop or conference) is organized for the donor, the national government, partners and other stakeholders regarding the results of the evaluation as well as possible follow up. The report is equally made available to other offices, Headquarters and partners for future sharing of best practices.

 

Concrete actions for follow-up on implementation of the recommendations are discussed with the actors for whom the evaluation was conducted.

 

100 See Norms and Standards for Evaluation (UNEG, 2016).

101 Quality review checklists for Evaluation ToRs and Evaluation Reports are available at the IOM Evaluation Webpage/technical references.


Source URL: http://uat.reintegrationhb.iom.int/annex/annex-4-monitoring-and-evaluation-tools