External Evaluator

From 2014-2020, Lembaga Studi dan Advokasi Masyarakat (ELSAM) together with Peace Brigades International (PBI) implemented a program to support human rights monitoring and advocacy work in conflict-prone areas of Indonesia. The program was centered in a capacity-building course for human rights defenders, the Basic Course for Human Rights Defenders, which operated from 2015-2018. From 2018-2020, the program conducted a variety of follow-up activities to consolidate the skills transferred during the Basic Course for Human Rights Defenders. The program was supported by the Civil Peace Service (CPS) program. During the four years of operations, the Basic Course trained 49 participants from seven different provinces, in a program that included 4 months of sessions in Jakarta with a wide range of mentors and guest lecturers with academics, civil society organizations, the diplomatic community and government bodies, followed by three months of field work. The trainings were based around four modules covering (1) human rights theory and legal systems; (2) safety and well-being; (3) monitoring and documentation; and (3) advocacy. Alumni of the program were further engaged in follow-up activities including a Training-of-Trainers and organizing their own trainings; participation in national and international advocacy work; and the publication of two volumes of individual research findings and a joint research project completed in 2020. Evaluation of the project The project itself was planned based on extensive evaluations and needs assessments of Papuan and national civil society during the period 2011-2014, and consistently conducted internal monitoring and evaluation engaging both of the implementing organizations; as well as the participants and other stakeholders. An external evaluation was conducted in 2018 covering the period from 2015 to early 2018, when five of the six cohorts had completed their training. A short external evaluation was also conducted to cover specific advocacy and protection work conducted in 2019. We are seeking an external evaluator to review the period of 2018-2020, including drawing some conclusions about the overall effectiveness of the capacity building program. This external evaluation will follow the standard of Project Relevance-Effectiveness-Impact-Sustainability, including an examination of project management; M&E tools in place; and lessons learned for future initiatives. Goal and objectives of the external evaluation Goal: An independent external evaluation will be carried out after the completion of the program to evaluate the effectiveness of the program in terms of its objectives. Objectives: This forthcoming evaluation is an opportunity to focus on the accomplishments and challenges to identify strategies to be developed for improvement. The purpose of the evaluation would include recommendations on future activities to build on the foundation created by this program. Deliverables:

1. The evaluators will prepare an evaluation report in Indonesian, that describes the evaluation and puts forward the evaluator’s findings and recommendations. (A draft report will be made available beforehand to ELSAM-PBI with a week period to provide inputs before finalizing the report).
2. A verbal presentation and explanation of results will be made by the evaluator to the project coordinator on request, if this request is made within one month of receiving the written report.

Key issues The evaluation should address the following key questions: Relevance: The extent to which the project is suited to the priorities and policies of the target group, the organisation responsible for the project. It is useful to consider the following questions:

• Is the program valuable to you?
• Do you find the program helpful or useful in your work?
• What value does/did the project add to the work of civil society in your area?

What will you do differently? Effectiveness: A measure of the extent to which a project achieves its objectives as laid out in the project contract. It is useful to consider the following questions:

• To what extent were the objectives achieved or are likely to be achieved? What information is available with regard to the agreed indicators that will help answer this question? What other information is available with regard to the achievement of the objectives?
• What were the major factors, such as the learning behavior, influencing the achievement or non-achievement of the objectives?
• What are the unintended effects of the program?

What were the major factors influencing the joint management of the project between ELSAM and PBI? Efficiency: A measure of the relationship between outputs and inputs.

• What are the inputs from program participants such as organizing meetings and debates or other activities?
• Are the stated objectives of the program being met in a timely manner with the available human and financial resources?

How well ELSAM and PBI able to perform together? Possible areas to consider may include e.g. management and administration systems, communication structures, and appropriate PME system. Outcomes and impacts: The positive and negative changes produced by this project, directly or indirectly. The evaluation should be concerned with both intended and unintended outcomes and impacts.

• How do the participants feel about the program/ its services?
• How many people have been trained?
• How many trained people have been working as local trainers or researchers?

What is the impact of the program in the knowledge and behaviour of the participants? Sustainability: Is concerned with measuring whether the benefits of a project are likely to continue after funding has been withdrawn. It is useful to consider the following questions:

• Is the programme compatible with goals and objectives of its local partners?
• Have local partners been able to benefit from the program activities, by improving their services?

To what extent will the participants and partner organizations be able to continue benefiting from the activities now that the program has concluded? In carrying out the evaluation, the consultant (s) will be expected to work with the Program Team to review and revise the program evaluation questions as appropriate, particularly in relation to questions on cross cutting issues. Methodology The scope of the evaluation includes the following:

1. The evaluators will review and analyse existing project documents in Indonesian and English provided by ELSAM-PBI.
2. The process of evaluation will include conducting of interviews and focus group discussions with representatives from the participants (trainees and sponsoring organizations) and collaborating agencies with a focus on civil society but possibly incorporating some government bodies. We anticipate these activities to take place in Jakarta and Jayapura, but will expect a majority to be done online due to the pandemic situation.
3. Evaluation will be both quantitative and qualitative in nature, examining the effectiveness of the broader strategy and the practical effectiveness of the tactics used to implement the program.
4. The evaluation will include a project management component. This will include looking at the internal coordination between the two organizations, the management of human resources, decision making processes, and financial management.

While maintaining independence/neutrality, the evaluation will be carried out using a participatory approach, which seeks the views and assessments of all parties. The evaluation will make use of the results of previous internal evaluations of this program. The evaluation team will use both quantitative and qualitative methods to evaluate the outcomes and impact of the program including its achievements and challenges. For this to be effective the evaluation process will play out in two different phases: Phase 1 – Desk study:  a review of the documents, preparation of evaluation questionnaire including methodology and data collection tools. ; and Phase 2 - Investigation: a collection of information both quantitate and qualitative using face-to-face semi-structured interviews and focus group discussions. Evaluation Time Frame The evaluation will take place during February-March 2021 (6 weeks in total) with a total of 20 working days. Evaluation Team and mission We are interested in bids from qualified evaluators working as individuals or in small teams, with a particular preference for those located in Jakarta and/or Jayapura. The combined qualities of the evaluator team should include:

• 5 years experience in capacity building projects
• Significant experience in human rights and field research
• Understanding of the civil society and human rights situation in Papua and nationally
• Ability to travel and conduct FGDs in Jakarta and Jayapura, or willingness to collaborate with local contacts to conduct FGDs online
• Knowledge of programming and project cycle management
• Experience as an evaluator of projects; experience in evaluation of projects conducted in partnership by more than one organization desirable
• Excellent written and verbal communication skills
• Experience in producing high quality written reports
• Understanding of English (to read internal documents and conduct interviews with some stakeholders)
• Ability to adhere to deadlines
• Authorization to work in Indonesia

Members of the evaluation team should not have been directly involved in the design and/or implementation of the projects. Interested parties should submit their bid to admin[at]elsam.or.id by 12 February 2021.