The Boundaries of Evidence in Conflict Management and Peacebuilding
By Jeroen Adam; December 11, 2013,
This blog was originally posted on the Asia Foundation Blog.
In The Asia Foundation’s recent report, “Subnational Conflict and International Development Assistance,” the authors argue that a sustainable solution to the many subnational conflicts plaguing different countries in Asia will ultimately depend on a true political transformation.
As explained by the authors, subnational conflicts are rooted in entrenched horizontal inequalities as minority ethnic populations are systematically excluded from access to political power. As explained by the authors, subnational conflicts are rooted in entrenched horizontal inequalities as minority ethnic populations are systematically excluded from access to political power. It is this dynamic which then feeds into the development of armed movements struggling for the rights of these minority groups. Therefore, it is argued, adjusting these historically grown power imbalances should be at the core of any sustainable solution to these conflicts. Based on my own research experience in what is a textbook example of a subnational conflict, namely the decades long armed struggle by the Muslim minority population on the island of Mindanao, the Philippines, I can only endorse this argument.
As such, my concern is not about the accuracy of this claim. Rather, I would like to say a few words about the boundaries of evidence-based programming in conflict management and peacebuilding. As any social scientist will acknowledge, politics and power are elusive and confusing concepts to study. As such, the argument as stated above raises a whole set of questions on how to map, assess, and measure long-term political transformations and how these have helped enable a climate of peace and security. It is also a challenge detecting to what extent certain programs have contributed to these political transformations. This analysis can only be undertaken through a careful combination of qualitative and quantitative data gathering based on long-term research stays, preferably in different localities. This is a time and money consuming exercise that requires well-trained researchers and puts a heavy burden on any organization. This is all the more the case in subnational conflicts which, as duly acknowledged in The Asia Foundation’s report, tend to be messy and complex as power constellations are fluid and the relationship between armed groups and civil population context-dependent. Inevitably, all this will divert resources away from classic conflict development programs.
Collecting good evidence demonstrating how a certain program has made a difference in terms of peace and security is a difficult and costly exercise. This became evident when conducting research on the usage of what the development community refers to as “theories of change” within the framework of an ongoing research collaboration between the London School of Economics’ Justice and Security Research Program (JSRP) and The Asia Foundation in the Philippines. [Read more about this collaboration in a blog post by the Foundation's Matthew Arnold] As could be derived from the interviews with Foundation staff, the introduction of a theory of change framework by DFID was greeted as a welcome space for renewed discussions about how and why change happens, as people were forced to systematically elaborate on truth claims. This exercise inevitably opened up a lot of methodological challenges. For instance, although it was acknowledged that settling the conflict in Mindanao will ultimately be a political exercise in the long term, the actual programming within the Conflict Management Unit of the Foundation in the Philippines came down to the quick settlement of local violent disputes through the deployment and mediation of strategic coalitions. This is a perfectly legitimate objective. Moreover, this allowed staff to come up with real quantitative data on the number of disputes settled. Yet, even in this rather minimal exercise, a whole set of new methodological issues emerged such as the question whether the settlement of a dispute could solely be attributed to Foundation interventions, how certain scales of contribution can be measured, and when a dispute can actually be considered as settled. In other words, what is indicated here – and this is an important point – is that there are boundaries to what can be measured and that there will always remain spaces of uncertainty and doubt.
In my opinion, evidence-based programming can only work when this concurs with a transparent and open communication about the boundaries of what can be proven in these messy subnational conflicts. As documented in the Foundation’s report on subnational conflict and also systematic literature reviews conducted within the JSRP, is that all types of organizations easily make bold claims for which there exists only limited data at best. For evident reasons, this is problematic. On the other hand, the legitimate demand for strong evidence should not result in the drive to design conflict management programs that prioritize the ability to produce neat, data evidence reports, at the sake of programs that have lasting, real impact but cannot be counted as effectively. Evidence based programming can in no way restrain us from asking and targeting difficult, yet unavoidable questions about power and politics. Therefore, while acknowledging the necessity for strong evidence, there should remain enough room to maneuver (and possibly fail) in dealing with the political ambiguities in everyday programming. This implies that organizations should be given the necessary confidence so that they can communicate transparently about the boundaries of what can be proven about their specific contribution to these long-term transformations.
Succinctly critiquing The Asia Foundation’s report on “The Contested Corners of Asia: Subnational Conflict and International Development Assistance,” the author argues that a political transformation that only can address the crux of subnational conflicts in South and Southeast Asia is too diffuse to be measured, and hence this attests to what the author calls the “boundaries of evidence-based programming.” Furthermore, as politics and power elude measurability, a whole set of questions unravel: how the political transformation itself can be measured; how its impact on peace can be ascertained; how the impact is to be attributed to any particular program while many other organizations might be simultaneously operating in the conflict locale. The author also cautions that in preponderating quantifiable data, organizations may likely incline to design programs in ways that focus merely on concomitances at the expense of addressing the systemic causes of entrenched subnational conflicts.
The article thus lays bare the challenges of designing evidence-based effective programming that can address the moot questions of identity and political empowerment when the reality is that donors demand concrete, quantifiable results. In my opinion however, “elusive” though politics and power are, the transformations that take place in those realms as a result of programming and the climate of peace engendered thereof can still be measured and discerned both qualitatively and quantitatively in that the institutions that embody politics and power are assessable and measurable. Legal changes guaranteeing the representation of minoritized, scapegoated ethnic groups in the body polity, military, bureaucracy, and the other state apparatuses can, for instance, be measured. Similarly, the author agrees to the notion that there exist political power imbalances and entrenched injustices. Based on my observations of the armed conflict in Nepal, if we believe that imbalances and inequalities can be measured; by the same token, the prevalence of peace, equalities and power balances resulting from programming can also be assessed qualitatively and quantitatively, though the process saps tremendous resources and time.