Mercy Corps is coming together with several organizations and donors with strong negotiation and mediation training programs in a unique and innovative initiative to collaboratively develop simple yet robust monitoring and evaluation tools that will better measure the impact of these particular programs. This initiative builds on the tools that Mercy Corps developed in our own programs. Given what we learned and challenges we faced in the process, we expanded the tool development to a core group of organizations who are committed to testing the tools in their programs. The collaborative nature of the work in which practitioners and donors share results and new ideas with regards to tool revision, is a basis on which the peacebuilding field can begin to answer difficult questions about the true impact of our work.
Mercy Corps Negotiation Tool Development Experience
Mercy Corps’s interest in this initiative comes from an experience we had while evaluating our conflict management and governance program in Iraq, which aimed to help influential Iraqi leaders gain the skills and support they needed to address a wide range of tensions and conflicts. During this process, we made a concerted effort to analyze the tools we used to collect real-time data throughout the length of the program. We used several tools at the program start, including (1) a 53-question participant baseline survey to gauge levels of expertise and perceptions of security, peace and trust in traditional and government dispute resolution and (2) a dispute database to collect and monitor key information about the disputes the participants were resolving.
Throughout the process of working with the field team and writing up the evaluation, we found several areas in which we could improve. For example, only 5 or 6 of the 53-question baseline survey were linked up to the program’s underlying theories of change (TOCs). Since analyzing survey results is time consuming and field staff has very limited resources, we wanted to make a distinction between what would be nice to know and what we need to know. Another issue we discovered was how difficult it was to draw out commonalities and trends among the dispute types, identities involved and results of the negotiation agreements. We found that being more consistent with data entry and narrowing down the disputes’ descriptive categories would help improve our effort.
The Iraq team, like so many others in different countries, went to great lengths to collect data and systematize the information from the trained participants. This effort cannot be taken lightly. So we asked ourselves: How can we refine and revise the tools so that they are user and field friendly while also successful at measuring the impact of our programs? This sparked a field-wide tool development process that soon involved input from other Mercy Corps’ teams in Nigeria, Pakistan, Myanmar, Jordan, Lebanon, Libya and Kenya. From there the effort to determine what key aspects should be captured by any negotiation or mediation skills training program began—not only in Mercy Corps’s programs but in larger peacebuilding community as well.
A Platform for Joint Tool Development: The Peacebuilding Evaluation Consortium
Mercy Corps’s role in the Peacebuilding Evaluation Consortium (PEC) is the platform from which we launched the tool development and new workshop series described in more detail below. Made possible by the support of the Carnegie Corporation of New York, the PEC is the first consortium of donors, practitioners and evaluation experts to jointly address the challenges of evaluation in peacebuilding. Together with the Alliance for Peacebuilding (AfP), the United States Institute of Peace (USIP), Search for Common Ground and American University, Mercy Corps seeks to build a system-wide architecture that supports the improvement of M&E practice in the peacebuilding field through a three-pronged approach: (1) Developing methodological rigor; (2) Improving the culture of evaluation, transparency and shared learning; and (3) Influencing policy on peacebuilding.
First Collaborative Negotiation-Mediation Evaluation Workshop
USIP hosted the first Negotiation-Mediation Evaluation Workshop that took place on December 13th, 2013. With the support of AfP and USIP, Mercy Corps brought together practitioners and donors interested in improving monitoring and evaluation. The group had a good mix of implementers, M&E specialists, and experts from different NGOs and donor organizations with strong negotiation, mediation, and leadership training programs (see participant list at the end of this post).
The December event was the first in a series of workshops on tool development. As a first step, the workshop focused on uncovering and being explicit about the theories of change that underline negotiation and mediation programs. The event organizers facilitated an exercise during which the practitioners teased out the TOCs and then wrote them up for the whole group to discuss. This fostered a rich discussion on the diverse approaches to negotiation and mediation work as well as the varying degree of influence and impact these programs aim to have. One particular thread of discussion was about timeframes and how these constraints impact what can and cannot be collected and measured. While most of the participants in the room agreed that their efforts were part of a long-term process to reduce violence and build peace, we asked and debated how we can actually show any results in a limited two or three year program. This question gets at the heart of why this group is convening: to roll up our sleeves and face the challenge of measuring and showing results during the life of our programs and beyond.
Common Underlying Theories of Change
While understanding that each organization has a unique approach to their work, the group found similarities and key aspects among the programs that helped us create common TOCs. The three TOCs below draw drawn from the collective effort of the first session:
IF we provide training in [conflict resolution/negotiation/mediation/dialogue], THEN we will see the participants addressing communal challenges through increased [negotiation/mediation/dialogue] efforts.
IF we build relationships across lines of divisions and include marginalized groups, THEN we will see a change in trust and inclusivity among divided groups and an increase in joint problem solving.
(This third TOC builds off of the previous two:)
IF we bring together the powerful and the powerless, provide specific capacity training and build sustainable relationships, THEN a network of peacebuilders will emerge THAT impacts local perceptions of governance.
Conclusion and Next Steps
Tools need to be linked back to our underlying assumptions if we are going to collect the right data to test our hypotheses and ultimately our impact on the ground. The three TOCs are the beginnings of a body of work in the peacebuilding field that will help practitioners improve their M&E systems for negotiation and mediation programs. Stay tuned for an update on the second workshop that will be held on April 7th 2014. The group will revisit the common TOCs, and move onto developing indicators and tools.
Leslie Wingender is the Peacebuilding Evaluation Fellow at Mercy Corps, focusing on developing and testing monitoring and evaluation tools alongside various country teams engaged in peacebuilding work around the globe. She completed her master's degree in Conflict Resolution at Georgetown University and a bachelor’s degree in Peace and Conflict Studies from University of California-Berkeley.
I do training evaluation as well so am particularly interested in the methods this team will evolve. Is there a specific mailing list I can get on to follow the work? (Or will updates consistently be posted to dmeforpeace.org?) I'm happy to check back but I'd love to get regular updates if that's possible.
Also, feel free to reach out if you think there's anyway I might lend a hand.
Best of luck with this important project!
Thank you so much for your interest in this initiative! Your comment about following this work brings up a great point on how this effort can and should be shared. We are using the DM&E for Peace portal to post summaries after all of our meetings so that we share the conversation and debate to the broader Peacebuilding M&E field at large.
I think as we continue to meet and start to share tools and develop collaborative ones, we can start an interesting discussion on how and where they can be applied. One way to do this will be when we use this portal to hold a Negotiation/Mediation tool talk down the road. We will be publicizing that once we are nearer to that stage. I will keep you posted!