Design, Monitoring and Evaluation for Peacebuilding

You are here

Principles and Strategies for Overcoming Challenges to Evaluation in Situations of Conflict and Fragility

The challenges of conducting evaluation in situations of conflict and fragility have been well documented.  They have been widely experienced and understood and therefore are not outlined in this post. 

What is less widely known and understood are the principles and strategies for overcoming these challenges, which is the focus of this post.  These principles should be carried throughout the evaluation process.  There is no excuse not to apply them!  When applied carefully, they enhance the credibility, use and rigour of the evaluation process and its results.

The three principles and corresponding strategies are:

  1. Conflict analysis
  2. Conflict sensitivity
  3. Evaluating effectiveness and conflict sensitivity
 

Conflict Analysis

Conflict analysis—which includes analysis of the political economy, stakeholders, and conflict drivers and causes—is central to any evaluation of donor engagement in situations of conflict and fragility: it provides an analytical framework for assessing the relevance, effectiveness, and impact of peacebuilding activities.

Conflict analysis can be used as the basis for assessing:

  • Whether activities (and the theories, strategies, etc., from which the activities are derived) have been sufficiently sensitive to the conflict setting;
  • Determining the scope of the evaluation (what will be evaluated); and,
  • Identifying pertinent evaluative questions.

As the basis for evaluative analysis, conflict analysis is a key aspect of conflict prevention and peacebuilding evaluations regardless of the design and methods used.

Hot Resources!

The Guidance on Conflict Analysis: Extraction of Key Points

Conflict Sensitive approaches to development, humanitarian assistance and peace building: tools for peace and conflict impact assessment, Chapter 2 by Saferworld

Peace and Conflict Impact Assessment Tip Sheet by SDC

Conflict Analysis Tools by SDC

Manual for Conflict Analysis by SIDA

Conflict Sensitivity

Conflict sensitivity, a key theme of this guidance, refers to the ability of an organization to:

  1. Understand the context in which it is operating;
  2. Understand the interaction between the intervention and the context; and,
  3. Act upon that understanding in order to avoid negative impacts and maximize positive impacts on the conflict.1

All activities in fragile and conflict-affected states must be conflict sensitive.  Negative impacts on a conflict may occur as a result of how a project’s outcomes contribute to peace or affect conflict, or from the operational aspects of engagement: i.e., how, where, and when donors and agencies operate and how they implement and distribute aid.

Just as a policy or intervention should be conflict sensitive, so too should the evaluation process itself.  As evaluations are themselves interventions that may impact conflict, it is important to understand that questions asked as part of an evaluation may shape people’s perceptions of a conflict.  Questions can be posed in ways that reinforce distrust and hostility towards ‘the other’.  The operation of the evaluation may also have negative effects: the way the evaluator (and enumerators for that matter) acts, the implicit and explicit messages and values they transmit may affect the degree of risk.

The evaluation process, too, should be conflict sensitive. A number of evaluators who contributed to the guidance spoke of incidents where someone they had questioned in the course of the evaluation had been arrested or otherwise threatened. Measures to avoid this include:

  • Redaction/non-inclusion of names of local members;
  • Reliance upon local/national teams who are able to more easily travel in dangerous zones – though their safety must also be protected.

The potential for risk in the evaluation process should be identified at the outset of the process. Doing so is the responsibility of evaluation commissioners and team leaders, and is a requirement of conflict sensitive, ethical evaluation.

A thorough, up to date understanding of the conflict is the first step in a conflict-sensitive evaluation process. The evaluation report must explain what measures were or were not taken to ensure the conflict sensitivity of the evaluation itself and any impact that taking or not taking them may have had on the results of the evaluation.

Hot Resources!

Conflict Sensitivity Website

Do No Harm Handbook by Mary B. Anderson

Fragile States Principles by OECD

How To Guide to Conflict Sensitivity by the Conflict Sensitivity Consortium

Evaluating Effectiveness and Conflict Sensitivity

Being conflict sensitive and evaluating conflict sensitivity are two imperative dimensions of evaluating conflict prevention and peacebuilding work. Evaluators can help assess whether or not the standard of conflict sensitivity has been achieved – as well as provide insights on how to improve sensitivity.

Conflict sensitivity does not automatically deliver an effective peace programme or policy. A conflict sensitive intervention is not necessarily effective in addressing the key drivers of conflict and fragility. Nor are explicit peacebuilding interventions necessarily conflict sensitive.

In assessing conflict sensitivity, it is important to look at the extent to which the intervention aggravates or mitigates grievances, vulnerabilities or tensions. For interventions that do not have an explicit peacebuilding goal, evaluators would assess the effects of the development or humanitarian outputs and outcomes on the drivers of conflict and fragility, e.g., infrastructure development, a more operational police or judicial system, etc.

All activities, whether explicitly aimed at peacebuilding or not, should be examined to assess their conflict sensitivity. The Do No Harm Framework2 identifies five ways in which operational components of an intervention may affect a conflict:

  • Theft/diversion: fuelling the conflict with stolen or diverted goods/funds;
  • Market effects: changing local markets with an influx of outside goods;
  • Distribution: distributing goods along the lines of the conflict;
  • Substitution effects: replacing existing functioning systems or structures;
  • Legitimization: giving legitimacy to a group or leader by working with them.

It also identifies four ways the behavior of agencies sends messages to reinforce the modes of warfare or, alternatively, non-conflictual relations:3

  • Conveys respect or disrespect to people and communities;
  • Communicates an agency’s willingness or unwillingness to be held accountable;
  • Treats people in ways that are perceived as fair or unfair;
  • Demonstrates transparency or lack of transparency.

Evaluators examining conflict sensitivity, a fundamental practice of evaluation in settings of conflict and fragility, may need to examine the evaluand’s own ways of working to determine whether the intervention is conflict sensitive. This would include examining inadvertent impacts of decisions about:

  • Staffing;
  • Criteria for selection of beneficiaries;
  • Selection of local partners;
  • Relations with local authorities;
  • Processes and procedures for distributing aid.4

Donors and agencies should not abandon their criteria or redistribute aid, but they must be aware of unintended conflict effects and develop options within the programme to mitigate them.

Hot Resources!

Conflict Sensitivity Website

Do No Harm Handbook by Mary B. Anderson

How To Guide to Conflict Sensitivity by the Conflict Sensitivity Consortium 

  • 1. CDA, Reflecting on Peace Practice Participant Training Manual, 2009.
  • 2. Mary B. Anderson, Do No Harm: How Aid Can Support Peace or War, Lynne Rienner, London, 1999.
  • 3. Anderson, Do No Harm, 1999.
  • 4. Anderson, Do No Harm, 1999.

Interesting post. I was rather intrigued by the following in the section on Conflict Sensitivity:

"A number of evaluators who contributed to the guidance spoke of incidents where someone they had questioned in the course of the evaluation had been arrested or otherwise threatened. Measures to avoid this include:

  • Redaction/non-inclusion of names of local members;
  • Reliance upon local/national teams who are able to more easily travel in dangerous zones – though their safety must also be protected.

The potential for risk in the evaluation process should be identified at the outset of the process. Doing so is the responsibility of evaluation commissioners and team leaders, and is a requirement of conflict sensitive, ethical evaluation."

What interested me about this section was how narrow-sighted it was. In many areas in conflict, protecting a respondent's identity or reliance on local/national teams will not decrease the possibility of threats, arrests, death, or disappearance. Often it is being seen with outside groups or in the presence of people who are known to work with outside groups or espouse contradictory idealogies that increases threats to particpants and team members (either foreign or domestic). I agree that the potential risks should be included as part of the process of planning to enter a conflict zone, yet it seems as if the potential risks to team members and participants are under analyzed. If the above section I quoted is indicative of the level of risk analysis and protection that goes into project preparation or analysis team preparation, I am surprised that more harm in not done to team members or participants. With many conflict participants becoming better acquinted with high-level technological applications to promote conflict, track individuals, and generate intricate and often clandestine networks, shouldn't we alter and enhance our risk analysis and protection efforts for teams and participants to account for complex and more potential sources of risk?

On the topic of Conflict Analysis as it exists to overcome challenges of evaluation in situations of conflict and fragility – as part of this analysis, are you referring to or is there room for a Conflict Vulnerability Assessment (CVA) in the process? My understanding of CVA’s is rooted in a graduate course where I produced a CVA about Egypt and the conflict between the Muslim Brotherhood and the Coptic Christians – a highly divisive conflict. Assessing vulnerability is critical in designing early warning systems for conflict and their possible interventions and preventions. This approach to analysis is informed by social-science theories of root causes of violence originating from structural variables, economic factors, discrimination, material drivers, and the role of government institutions. This approach seems comparable to what the article is suggested in the sense it seeks to analyze similar categories. It also seems the ultimate goals align, as they each tool sees conflict prevention as a key factor in peacebuilding evaluations.

@Nfuhr: Thank you for your thoughtful comment. I agree with your observation that potential risks should be included as part of the process of planning to enter a conflict zone. From my experience, while some organizations do take measures to protect respondents and local team members, they are not frequently reported out externally, which is unfortunate. If you do come across resources that discuss this in detail, I would encourage you to please share them with the DME for Peace community.  The purpose of Jonathan's post was to give a broad overview of the principles one must adopt when conducting M&E in fragile states.  

@Jordan: I'm glad you brought up Conflict Vulnerability Assessment (CVA). While there is certainly some overlap between Conflict Analysis and CVA, I do think they are important differences between the two and therefore, they are not interchangable. I believe CVA is used more to assess how vulnerable a particularly entity is to violent conflict (capacity of the entity to cope with risk factors, manage tensions etc.) while the scope of conflict analysis is broader. I imagine conflict analysis to be a more generalized study of the profile, causes, actors and dynamics of the conflict.