Shifting the Power Project - End of Project Evaluation: TOR

This evaluation is to fulfil the project’s accountability to its primary stakeholders – the local partner organisations – and to its secondary stakeholders – the project consortia and donors (INGOs, Start network and DFID). It seeks to contribute to the DEPP evaluation being conducted by the Harvard Humanitarian Initiative and its key findings and learning are intended to be published and shared to the wider humanitarian community.

The overall objective of the evaluation is to assess the project’s theory of change, that “a shift of power towards locally owned and led responses will contribute to a more balanced humanitarian system that delivers more effective and accountable humanitarian response.”

I. Background

The current international humanitarian system, dominated by large international organisations, is being stretched to its limit. Dealing with growing frequency, unpredictability and complexity of emergencies, it appears increasingly unfit to deal with these challenges, let alone address future ones. One opportunity to address these is getting the balance right between international and local response. Local actors are the first and main responders after a disaster strikes: they know the context and speak the language; they are there, and will stay there long after international actors have left. A growing body of evidence demonstrates that partnerships with national and local organisations enhance the relevance, appropriateness, accountability and connectedness of humanitarian responses, and ensure better linking up of relief, rehabilitation and development. Despite this, the current system favours working with large international actors, who frequently ignore local and national actors.

Re-establishing the balance is the key, toward a more balanced humanitarian system where local actors take their place alongside international actors. A shift of power towards locally owned and led responses will result in more effective and accountable delivery of humanitarian assistance to disaster affected communities.

Six international organisations – ActionAid, CAFOD, Christian Aid, Tearfund, Concern and Oxfam – are working together in the Shifting the Power project, supporting 55 of their local and national NGO partners who share the vision and ambition of playing a leading role in decision making and responding to crises in their countries. The project is being implemented until March 20181 in 5 countries (DRC, Ethiopia, Kenya, Pakistan and Bangladesh) and is a START Network project supported by the United Kingdom’s Department for International Development (DFID) through the visionary Disasters Emergency Preparedness Programme (DEPP), an ambitious suite of projects launched by the START Network to develop decentralized approaches to capacity building and to improve the quality and speed of humanitarian response.

The aim of Shifting the Power is to support local actors to take their place alongside international actors in order to create a balanced humanitarian system. To do so, it works around 3 main areas:

1. Strengthen local and national organisational capacity for preparedness, response, decision making and leadership in humanitarian response

2. Support local and national organisations to have greater representation, voice and recognition in their relevant networks and platforms,

3. Work with the consortium member INGOs for them to recognise and respond to local/national organisations capacity, leadership & voice

A baseline data collection through an online survey of L/NNGOs and INGOs working in the five countries was conducted between June and September 2015 with a report published in December 2015. The baseline report provides an overview of the current state (capacity, networks and relationships) of the organisations involved in the project. It helped the project team to reach a better understanding of the humanitarian context in each country, including issues faced by L/NNGOs and INGOs, to measure changes achieved through the project activities, and to improve the over-all quality of the project.

The capacity strengthening component which involves an in-depth two year programme of support for L/NNGO partners started with the selection of the 55 L/NNGOs based on set criteria. A critical step has been for the organisation to fully commit and take ownership of efforts to strengthen their humanitarian capacity. The 55 local partners conducted a self-assessment using the SHAPE framework2 in a 2 to 3 day participatory workshop that involved a broad, robust and self-critical discussion with participants from throughout a partner organisation to support them in assessing their humanitarian capacity. After completing the capacity self-assessment process, partners developed Capacity Strengthening Plans (CSPs), prioritising 3-6 indicators from the SHAPE framework, crafting specific capacity strengthening objectives aligned to those indicators and identifying specific capacity strengthening methods. Implementation of the CSPs began at different times in the five countries between April and August 2016.

In May 2016, the first World Humanitarian Summit (WHS) was held and this led to a voluntary commitment of major governmental donors and international relief agencies, also known as Grand Bargain. One of the commitments was around “more support and funding tools for local and national responders” that has become known as ‘localisation’. The focused work and intensive debate on localisation that followed set the backdrop for the second half of the implementation of the Shifting the Power project.

Activities around the second work stream, L/NNGO voice and influence, also began in earnest in 2016 while the research, Localisation of Aid: Are INGOs walking the talk? was completed. Implementation of the CSPs is on track to be completed in December 2017 as well as other project deliverables on the second work stream, voice and influence, and the follow-up actions on the Localisation of Aid research.

II. Purpose and Objectives of the evaluation

This evaluation is to fulfil the project’s accountability to its primary stakeholders – the local partner organisations – and to its secondary stakeholders – the project consortia and donors (INGOs, Start network and DFID). It seeks to contribute to the DEPP programme evaluation being conducted by the Harvard Humanitarian Initiative and its key findings and learning are intended to be published and shared to the wider humanitarian community.

The overall objective of the evaluation is to assess the project’s theory of change, that “a shift of power towards locally owned and led responses will contribute to a more balanced humanitarian system that delivers more effective and accountable humanitarian response.”

In doing so, the evaluation will:

1. Assess the project against the OECD-DAC criteria of: relevance, efficiency, effectiveness and impact.

2. Assess the extent to the project has progressed towards its intended outcomes as given in project plans and against logframe indicators.

3. To address the project’s key learning questions (as below) and summarise lessons learnt from implementation in each country and globally, and, and make recommendations for key stakeholders as to how they can best continue to work towards the project’s goal.

The evaluation will seek to answer the following questions:

1. If and to what extent and in what ways have improved capacities, leadership, and voice of the 55 local and national organisations in five countries contributed to faster, better quality and more effective responses?

2. If and how have local/national NGOs through Shifting the Power influenced decision making of, improved access on information and resources, and undertake leadership roles in national humanitarian platforms and networks?

3. If and in what ways has the project addressed the barriers and challenges posed by international humanitarian agencies to support new roles and ways of working with local and national organisations?

4. To what extent are the project results sustainable beyond the life of the project? What and how local/national partners and INGOs can do to continue to work towards the project’s goal?

In addition to these questions, we would also like to examine the following:

5. How has the project contributed to strengthening the evidence base of what works in strengthening capacities, voice and influence of local and national organisations?

6. How has the project’s governance and management structure helped shift the power?

7. Have resources been used efficiently? In general, do the results achieved justify the costs? Could the same results be achieved with fewer resources?

III. Scope of the evaluation

The evaluation will cover all five countries (Bangladesh, Pakistan, Ethiopia, DRC, and Kenya) and the coordination work done at global/UK level. Direct project participants and partners (local/national partner organisations, International and Country-based Project Management Teams, International and National Steering Committees, and Technical Working Groups) will be involved. Other stakeholders such as government and UN agencies and other International and National NGOs that have been indirectly involved in the project may be consulted. International, National and Local NGOs that have not been involved may also be contacted if they can provide a useful counterfactual.

IV. Expected methodology

We are seeking a consultant(s) to lead us through a process that can address the above questions. Ideally, we are looking to adapt a participatory evaluation methodology where key intended users specifically the country-based project management and local/national partners’ staff are able to participate in the design and conduct of the evaluation.

The evaluation should consolidate, verify, and build on the findings and recommendations of the various learning reviews and other learning products that the project has produced. The Global Baseline carried out in 2015 will provide comparative points for this final evaluation. This included an assessment of local and national partner capacity using the SHAPE framework, and a survey and social network analysis of project partners that will be repeated at endline. These can be facilitated by country management teams but should be supported and analysed by the global evaluation consultant(s).

The evaluation methodology should include an analysis of both qualitative and quantitative data, and analysis should disaggregate results by sex, organisation type and country (as appropriate). These are some of the elements that are currently anticipated:

• Desk research that will include:

o Shifting the Power Baseline Report

o Getting into shape? A review of Shifting the Power organisational capacity assessment approach

o How has Shifting the Power influenced local and national partners' response to emergencies?

o Increasing the voice and influence of local and national NGOs

o Year 1 Overview

o Year 2 Overview

o Various case studies, researches and learning products from the five countries

• Consultation, formation and training of country evaluation teams (face to face in at least 2 countries and the rest to be done remotely)

• Interviews and discussions with key stakeholders and non-DEPP stakeholders including visits to at least two countries and remote engagement with three countries

• Telephone/Skype/Face to face interviews with UK-based stakeholders

• Workshop to present initial findings and draw up recommendations

V. Coordination, roles and responsibilities

A Steering Group will be formed to provide over-all coordination and support including signing off on key decisions. This will be composed of the International Project Manager and four members of the International Steering Committee and the Technical Working Group with at least two coming from the countries. The consultant(s) will report to the Steering Group while day to day management will be done by the International Project Manager. Field visits will be managed by the country project management teams while over-all logistics and administrative support will be coordinated by the Project and Communications Assistant.

VI. Expected outputs / deliverables

We are planning to begin this review in late November 2017 and it should be completed by early March 2018. The methodology and process for the review will be agreed with the Steering Group, and should include the following deliverables:

1. Workplan for the review which includes agreed review questions, stakeholders, proposed methodology and timeline

2. Inception report following meeting with Steering Group and other key stakeholders

3. Draft and final evaluation report of no more than 40 pages (excluding annexes). The report will include:

o Cover page (title of the evaluation report, date, and name of consultants)

o Contents table

o Executive summary of no more than two pages outlining the purpose of the evaluation, main points of analysis, key findings, conclusions and

recommendations

o Introduction outlining the background to the project and the evaluation

o Purpose and objectives of the evaluation

o Methodology, indicators used and limitations of the evaluation

o Major findings (data analysis, including gender analysis, and response to evaluation objectives and learning questions)

o Lessons learned and recommendations

o Annexes : Country reports or case studies, details of data collection tools, schedule of field visits and meetings, list of people interviewed, bibliography of key documents consulted; TOR for the evaluation, detailed data tables

The evaluation team are required to make a presentation of key findings and recommendations to the Steering Group and other key stakeholders (date and venue to be confirmed). The raw data (all transcripts, quantitative data, data collection tools) must be handed over to ActionAid together with the evaluation report.

VII. Expected timetable

(Please see attachment for details)

VIII. Evaluator qualifications

We are looking for a consultant(s) with extensive knowledge and experience of multi-country humanitarian programme/project evaluation and ideally with significant experience in evaluating humanitarian consortium projects that involve local and national organisations.

More specifically, applicants should have:

 Experience in humanitarian sector specifically on monitoring and evaluation, research and/or learning reviews

 Demonstrable understanding of the concepts and debates around localisation of aid and or locally-led and managed preparedness and response work

 Experience in qualitative methods and analysis including the use of theory of change in evaluations

 Practical experience in using power analysis tools and/or social network analysis

 Evidence of strong downward accountability mechanisms used with project stakeholders/evaluation participants to actively share results and learning

 Evidence of successful collaboration with NGOs, and particularly collaboration which included capacity building and on the job training with field office staff

 Evidence of client responsiveness creativity and flexibility of approaches towards clients’ needs and or challenges in conducting evaluations

 Fluency in English, and in addition, knowledge of relevant languages in any of the five countries is desirable

We would welcome applications from small teams of consultants, with a designated team leader.

IX. Budget

The budget available for this review is £40,000. Payments to the consultants will be made in pounds sterling. This budget is inclusive of all costs including in-country data collection costs that may be borne by ActionAid (where it is cost-efficient to do so and to be mutually agreed). This is inclusive of VAT or any other tax responsibility. Evaluators registered outside of the UK should note that the VAT element (20%) will be paid directly by ActionAid to the UK tax authorities and therefore will not form part of their payment.

X. Application process: submission of proposals

To apply, please submit:

 The CV(s) of the applicant(s)

 A covering letter or expression of interest -highlighting relevant experience and skills to the specific evaluation, and the reasons for interest in the work.

 A technical proposal with suggested approach, timeline, and itemized budget

 Examples or links to similar work /evaluation reports

 The names of at least 2 previous clients to contact for references.

Please send the above to Nikita.Samaratunga@actionaid.org by 20th November 2017. Interviews for shortlisted candidates will be held during week of 27th November 2017.