Design, Monitoring and Evaluation for Peacebuilding

You are here

Measuring Long-Term Change in Conflict Transformation

“One of the major challenges for design, monitoring and evaluation for peacebuilding is the timeline. Conflict transformation takes place over long periods of time—or it doesn’t in some cases. But we are often forced to look in two to three year blocks of time at projects, or sometimes if we are lucky at programming more broadly, and it’s very, very difficult to really know what type of results you are achieving in that period of time.” –Tom Bamat, Senior Technical Advisor for Justice & Peace at Catholic Relief Services

Conflict transformation is, by its very nature, a long-term process. This is one of the fundamental principles of conflict transformation put forth by scholar-practitioner John Paul Lederach in his many writings: the conflict transformation lens is specifically concerned with mid- and long-term social change. This is as opposed to the traditional conceptualization of conflict resolution, which is concerned with immediate resolution of a very specific problem.1

One often hears these principles espoused when listening to practitioners and scholars of peacebuilding and conflict transformation speak, regardless of the event or location. And indeed, this is frequently how practitioners and organizations practice peacebuilding and conflict transformation: long-term and, sometimes but not frequently enough, systemic change (a related topic, worthy of its own separate discussion at another time). 

The principle of long-term change begins to fall apart, however, when it comes to measuring and evaluating conflict transformation. Too often the standard is the immediate measurement within a project lifecycle, and understanding the true, long-term impact of this type of work. This may be because of a misalignment of principles and priorities between donors and implementers, and the methodological complexity of measuring and evaluating long-term change.

Hot Resource! Reflecting on Peacebuilding Evaluation Interview Series: Tom Bamat by the Learning Portal for DM&E for Peacebuilding

“It is still ‘easy to throw away our values when it comes to evaluation.’” – Fletcher School of Law and Diplomacy at Tufts University Staff Member2

Donor-Implementer Tensions

One of the fundamental barriers to long-term measurement of conflict transformation is donor and organizational policy on the primary purpose of evaluation: is it for accountability (and if so, upwards to donors, or downwards to beneficiaries?) or for learning and effectiveness (and if so, learning for whom)? Of course some sort of equilibrium must be reached and this will vary from project to project, donor to donor and organization to organization. Nevertheless, upwards accountability would seem to be frequently prioritized by both donors and implementing organizations in evaluation policy: both can and should do better.

Hot Resource! Starting on the Same Page: A Lessons Report from the Peacebuilding Evaluation Project by Melanie Kawano-Chiu

There is a real need for both donors and implementers to demonstrate the effects of projects, both for the further legitimation of the field and developing evidence of ‘what works’. Upwards accountability is important, but should not come at the expense of truly understanding the impacts of our work. In the cases of USAID and USIP, there is a real need to educate policymakers on the long-term nature of this type of work, and encourage the development of supportive policies that do not sacrifice the long-term for immediate short-term understandings.

Peacebuilding and conflict transformation funding is in a precarious situation, at least in the United States. Budgets devoted to addressing peace and conflict have been cut as a result of the ongoing economic crisis, including both USAID and USIP. As a result there is a heightened need to demonstrate impact—something our field already struggles with—in order to (continue to) receive funding (or, at the very least, outcomes). Given the methodological issues involved in such efforts, this frequently results in measurement at the project-level, with very little rigorous evidence to demonstrate outcome or impact at the conflict or context system levels.

It is important to stress here that this misalignment of priorities and principles is not solely caused by donors. There is much implementing organizations could do to begin moving towards long-term monitoring and evaluation of conflict transformation. This includes creating organizational frameworks for long-term monitoring of outcomes and impacts beyond project life cycles, the use of standardized indicators across the organization and across and beyond projects, and the development of staff capacities for such activities.

Methodological Complexities

The problem of measuring long-term transformations is not solely political: significant methodological hurdles must be overcome in order for it to become an operational reality. Such issues include the attribution versus contribution debate, and identifying a range of appropriate methods and methodological approaches to such measurement.

The attribution-contribution debate is not new in conflict transformation work. Long-term change, it can be argued, is fundamentally non-linear and complex3, compounding the challenge of attributing a specific change to a specific intervention by a specific actor. Many organizations talk about contribution analysis, but there are few major efforts to further develop this concept in the evaluation of conflict transformation—an activity perhaps for implementers, donors and academics, together.

Hot Resource! Contribution Analysis: An Approach to Exploring Cause and Effect by John Mayne

Related is the increasing use of impact evaluation as the “gold standard” in international development evaluation (to be further explored next week), and the debate over the most appropriate methods and methodological approaches to evaluating conflict transformation work. Further work remains to be done in this area vis-à-vis identifying the concrete benefits or added-value of one approach over another in a given situation and why—remember, the evaluation question and intended purpose of the evaluation should guide your choice of the most appropriate methods.

But much of the ‘low hanging fruit’ has already been picked: there exist a large handful of evaluation handbooks, some of which are specifically for peacebuilding and conflict transformation. “More needs to be done, but the existing frameworks and manuals are good enough, so that radically improving on them in the short term will be difficult.”4 So perhaps the challenge of measuring long-term change is simply one of adapting existing frameworks for longer time periods—and, of course, the allocation of enough supportive resources.

Moving Forward

The situation is not hopeless: practitioners of conflict transformation are inherently empowered with agency, an essential ingredient for altering the current status quo. And more than the presence of mere agency, concrete activities are currently underway to address ongoing dynamics in the evaluation of conflict transformation.

Perhaps at the forefront, at least here in the United States, is the Alliance for Peacebuilding. Their Peacebuilding Evaluation Project did wonders to place the issue of evaluation and its improvement firmly on the agenda of both donors and implementing agencies. And more than just dialogue, the Project is now seeking to effect policy of both types of actors for better evaluation policy, transparency and learning. These early efforts have been critical in laying the groundwork for further improvement in our evaluation practices, and, critically, opened dialogue on pertinent issues between donors and implementers on evaluation and peacebuilding practice in general.

At a more practical level, the work of CDA Inc. Collaborative Learning Projects’ Reflecting on Peace Practice (RPP) project has reasserted evaluative thinking and evaluation considerations throughout the lifecycle of a project, and began to bridge ‘traditional’ conflict transformation approaches with the emerging paradigm of systems theory and its applications. While its full application in evaluation is still emerging, RPP remains a promising evaluative methodology for long-term change because it inserts evaluative thinking into all stages of the program lifecycle. If used correctly, RPP can help programs chart long-term change and track how individual projects build off on another.

Hot Resource! Reflecting on Peace Practice Manual by CDA Inc. Collaborative Learning Projects

But the real value in altering the timeline of measurement for conflict transformation activities will emerge from the collective and cumulative impacts of various actors in coordination and collaboration. One might even think of it as conflict transformation for the field of conflict transformation.

Jonathan White is the Manager of the Learning Portal for DM&E for Peacebuilding at Search for Common Ground. Views expressed herein do not represent SFCG, the Learning Portal or its partners or affiliates.

This is an excellent tension to grapple with in peacebuilding evaluation. While the donor-implementer tension is a difficult one, I think it is overemphasized. In my experience with an international NGO, donors (and I speak to individual donors only as I have no experience with major foundations/grantors) rarely fund an organization solely on its ability to run a given program and produce specific results. Rather, they assess the organization’s vision or mission statement to determine if it matches the donor’s values and is thus worthy of donating to (and from there the capacity and aptitude of the organization is assessed). Therefore, the donor is already thinking long term and I would expect most donors would welcome some evaluation and reporting that speaks to long-term change, putting the onus on implementers to develop these tools.

Rather than the donor-implementer tension, I would argue the methodological challenges are the biggest hurdle to overcome. I’m curious to hear more on how systems theory might contribute to overcoming the attribution-contribution debate, as I think it has much to offer. Robert Ricigliano’s (2012) application of systems theory to peacemaking is especially helpful. Through the ability to see the whole picture, and how the various parts interact with each other, systems theory can help to evaluate a particular peace project’s place in the conflict system, and thus its effectiveness in creating change. Systems theory could also be applied through multiple time frames, in order to understand how an initiative might be positioning itself now for long-term impact in the future.

Thanks for raising this important issue!

Ricigliano, Robert. 2012. Making Peace Last: A Toolbox for Sustainable Peacebuilding. Boulder: Paradigm Publishers.

 

John Paul Lederach and other scholars-practitioners argue that a conflict transformation is a long-term process. Measuring the tangible impact of such a long-term conflict transformation and that too at a time when the entire NGO sector, bitter though it sounds, has turned into a kind of industry, is very complex and challenging. Donors seek results and the implementers tend to design projects in ways that target the outcome of conflict, rather than its root cause. The author in the present article has raised two significant obstacles to conducting the long-term measurement of conflict transformation: the donor-implementer tension and the methodological complexity.

I wish to look at the issue slightly differently here. Based on my experience and observation of the conflict dynamics in Nepal, I agree to the notion that “transformation” takes time, but it cannot be eternally postponed citing the theoretical reasons. For me, “transformation” is not a destination; it is rather a process. The day will never come when the peacebuilders could say that the long-craved transformation has finally happened. Two major issues, little emphasized even in the peacebuilding discourse, I argue, stand in the way to making that “transformation” happen, at least in the context of the developing world. First, there exists a big chasm between donors and implementers; only a nominal chunk of the allocated fund in many cases finds its way through institutionalized red-tape and corruption down to the grassroots. If an NGO working against witchcraft, for instance, gets funding from an international donor, a huge portion is spent in organizing seminars in deluxe hotels. Only a handful of elite gatekeepers benefit from this, let alone the women burnt alive in distant villages of the country. Second, what is also entirely ignored in the development discourse, I think, is the agency of the beneficiaries ‒ the agency of the intervened subject is largely undermined, and even the attribution-contribution discourse eschews the subject’s agency. This is where the error lies. There is an urgent need on the part of both donors and implementers to think of peacebuilding not as any other developmental activity. What's more, it is important for peace-practitioners and policymakers to conceive of peacebuilding not like any other profession, rather as a "vocation", which in Lederack’s words, is the [l]onging of the true home” (167, The Moral Imagination). Only then the projects can be anchored to the programs, which in turn help effect systemic change.