Design, Monitoring and Evaluation for Peacebuilding

You are here

Effectiveness of Inter-Communal Conflict Analysis in Burma 2016 by State Dept Bureau of Conflict & Stabilization Ops

Author, Copyright Holder: 
Bruce W. Hemmer, Ph.D., Bureau of Conflict & Stabilization Operations, US Department of State

From April to October 2016, the U.S. Department of State’s Bureau of Conflict and Stabilization Operations (CSO) deployed a senior conflict analyst to assist U.S. Embassy Rangoon with field analysis of subnational intercommunal conflict (ICC) dynamics throughout Burma.  He produced twelve cables on subnational dynamics, eight actor maps, a capstone nation-wide analysis, seven outbriefs, an early warning checklist, and diplomatic and programmatic recommendations.  Because Rakhine State already received considerable attention, the focus was primarily on other states deemed at risk, though some attention was paid to Rakhine.  To measure the effectiveness of this analysis at informing recipients (U.S. government officials in Rangoon and Washington) about ICC dynamics, and shaping and coordinating related diplomatic and programmatic interventions, CSO conducted two anonymous online surveys of analysis recipients.  (For independence, another CSO officer in a different office conducted and analyzed this survey.)  The results will inform CSO analytical practices elsewhere.  Furthermore, this approach to measuring the effectiveness of analysis may serve as a model for other efforts.

The baseline survey in early June 2016 was conducted before significant reports had been distributed by the analyst, and the endline in late Oct - early Nov 2016 was conducted one-two weeks after the final report was distributed.  The surveys were sent to 50 analysis recipients in Burma and Washington, DC.  Fifteen people responded to the baseline, with ten reporting a very strong or strong focus on Burma in their work, and five reporting lower levels of Burma focus (almost certainly people in Washington).  The endline was distributed to the same set of analysis recipients, but the resulting sample consisted of only nine people, all with either a very strong or strong focus on Burma.  Because the baseline survey showed the level of focus on Burma to affect answers on several other questions, the baseline was weighted to match the endline demographics in order to make valid comparisons in this report. See Annex B for details.  While such a small sample size would ordinarily be a concern, this is a high percentage of the officers in these agencies who focus on Burma, and hence is a reliable signal of how the most important analysis recipients were affected.  Affecting them can be expected to have secondary effects on others.

Comparing endline to baseline results shows substantial improvement on every indicator of analysis quality, learning about subnational ICC dynamics outside Rakhine State, and effects of the analysis on diplomacy, programming and coordination.  The average improvement across the indicators was 0.5 points, on a scale with maximum potential change of 4.0 points, or 12.5% of the scale length.  See Annex C for details of the questionnaire.

As doing M&E on conflict analysis is rare, this may serve as a useful example of one way of doing it. (Another means, not used here, is content analysis of later strategies, plans, diplomatic actions, or programs to look for language and concepts from the analysis and its recommendations.)