Design, Monitoring and Evaluation for Peacebuilding

You are here

Revitalizing Reflective Practice

Revitalizing Reflective Practice

Peacebuilding values and has a rich history of critical reflection by practitioners.  Examples include Beyond Neutrality by Bernard Mayer, The Moral Imagination by John Paul Lederach and Watching the Wind by Susan Collin Marks, to name only a few.  Lately reflective practice has not received the same attention, support and investment as professional external evaluation.  A number of possible explanations come to mind. 

  •  Reflection and evaluation compete for the same resources, namely time
  • Donors policy prefers external evaluation to self-evaluation
  •  General preoccupation with big results rather than incremental processes and nuance
  •  The project/program paradigm trumps the practitioner

Learning about how peacebuilding works inevitably requires, experimentation, reflection and evaluation.  How can we take better advantage of self-reflection and evaluation?  For innovations in complex contexts developmental evaluation offers concrete possibilities.  As Patton explains, “developmental evaluation is designed to be congruent with and nurture developmental, emergent, innovative and transformative processes.”  The same may be said for reflection.

What do we do, however, for more established and less innovative initiates?  How do we preserve a rich tradition of self-reflection and take advantage of what evaluation has to offer?  A few initial thoughts include;

  • Make better use of case studies and narratives
  • Facilitate and document more local practitioners’ reflections
  • Recognize and label the “valuing” done during self-reflection and evaluation
  • Use self-reflection as an input into evaluation
  • Use evaluation to validate reflections
  • Have evaluators observe reflections

What ideas do you have?  How do we keep these two rich traditions from becoming accidental adversaries?

 

Your post is very thoughtful and I find this trend troublesome. I was first introduced to peacebuilding through authors such as John Paul Lederach and Johan Galtung. My understanding and practice of peacebuilding is therefore intimately linked with reflection. I also cannot help but observe a "professionalization" of peacebuilding--a formalization of workplace processes and environments, etc.; not that this is a bad thing, in many ways it can be good, but my point is that as we become more "professional" we may loose touch with "the art and soul of building peace." Both offer important benefits to our practice and we need to find a balance. 

But I also wonder to what extent the decrease of reflection is confined to the Global North and/or INGOs? I previously worked with an organization based in Cambodia, Centre for Peace and Conflict Studies, that convenes a yearly conference for peacebuilders in Asia (mostly of Oriental origin; Action Asia Peacebuilders Forum) with the express purpose of reflection and revitalization. Each year a broad theme is selected, and practitioners working in, on or around that theme are invited to present and reflect on their work and results. These are often well attended, several hundred individuals. You can find their most recent publication, from the 2010 Action Asia Peacebuilders Forum, here. Other organizations, such as the network ACTION Global, also support reflective peacebuilding. Perhaps this is a slight deviation from your primary point, that reflection is increasingly being replaced by formal external evaluation, but I feel it is nevertheless important to raise: reflective peacebuilding is still alive. The caveat of course being that more could and should be done to support this critical exercise, and, those reflections need to be shared! 

I believe the issue of reflection is also linked to learning: we reflect in order to learn. To extrapolate this to field-wide dynamics, we require field-wide systems to support reflection, learning and their dissemination and utilization. The Alliance for Peacebuilding and the Global Partnership for the Prevention of Armed Conflict seem to be excellent vehicles to support such systems. Perhaps the upcoming AfP Conference in the first half of 2012 will provide an institutionalized space within our professional field for active reflection. 

Mark, I understand you often conduct evaluations. Have you ever included practitioner/project staff reflections in your evaluation design and if so, how was this done and who initiated the process (was it evaluator or evaluand initiated)? Did this add any extra time or costs to the evaluation budget? The purposeful inclusion of reflection in formal evaluation designs could help prevent formal evaluation and reflection from becoming, as you say, accidental adversaries, and indeed, the two could be quite complimentary to each other. 

Dear Jonathan,

Thanks for your thoughtful post.  I couldn’t agree more.  Reflection indeed continues and is alive.  The Action Asia Peacebuilders Forum you referenced is a powerful example.  Thank you.

You asked about my evaluation practice.  In the few evaluation on which I have had the honor of serving there were several obstacles to integrating prior reflections into the evaluation process. 

  • Safety - To one degree or another reflection, peacebuilding and evaluation require safe spaces.  These are difficult to establish when the evaluator is a foreigner who is only on site for a few weeks.   
  • Synchronization - Reflection seems to happen when one is either in the thick of it or entirely separated with the benefit of time and distance.  This is to say reflection often happens as interventions play out whereas evaluations are all too frequently after thoughts.
  • Documentation - Where peacebuilding teams have established a reflective practice, its work is rarely documented making later use difficult.  It some places documentation of this nature is dangerous.

These are all surmountable with adequate planning, anticipation and resources.

While I agree that balance is in order, I’m also interested in how to blend the two for the purpose of learning.  If we understand reflective practice to be an evaluative approach, what other evaluation approaches add value and in what ways?  For example reflection and developmental evaluation go well together.  What else? 

Does exposure to and the application of evaluative thinking help peacebuilding practitioners deepen their reflection?  Given SFCG’s commitment to evaluation, are examples of evaluative thinking showing up in the reflections of SFCG practitioners or partner practitioners?

I also wonder if we need to do a better job of legitimizing and advocating greater donor acceptance of findings from reflective practice.  USAID’s evaluation policy virtually requires the use of externals.  Given this constraint, how can externals make better use of practitioners’ reflections?  Although demonstrating results is important, peacebuilding is still so young that demonstrating learning ought to be the bigger priority.

I also posted this discussion on LinkedIn's Conflict Transformation, Peacebuilding and Security Group. The following discussion ensued:

Amjad Saleem • i wouldn't say tension, but i think the question that often comes up is the question of time, to allow accountability issues and the quest for justice to take shape. in the form of restorative justice, i think reflection allows us that space and time to allow accountability issues to take their natural step and so it should become part of the formal process in terms of linear steps of peacebuilding

Koenraad Van Brabant • Reflective ways of working should be normal, as peacebuilding takes place in uncertain and changing environments, and is in reality more a question of 'navigating' rather than walking a logframe on cruise control. The excessive focus on 'the project' tends to distract however from 'navigating' a socio-political environment, because the incentives push us to be concerned about 'implementing the project (as per the terms of the contract) rather than being adaptive to remain 'relevant' (a critical evaluation criterion) and if need be change tactics so as to retain the potential to be 'effective' (another evaluation criterion). Experience tells us that those who live and work 'in the midst of it' can be very smart/savvy but don't typically have the privilege of taking some distance and look at the broader picture without the clutter of endless daily details. Period retreats - perhaps supported by someone who understands the context and the programme but who also comes with fresh eyes- and period strategic reviews create time and space for structured and deeper reflection. If peace practitioners operate in such reflective manner, they don't have to feel concern about a 'formal' evaluation, and will have anticipated many of the important questions that a formal evaluation may ask - and have solid answers ready. Then the question is what the main purposes of the formal evaluation are: accountability (to the donor or to stakeholders in the programme, and if so, accountability against the terms of the contract or with regard to remaining relevant and potentially effective which may have required changes from the contract) or learning and strategic management. The latter two reinforce the reflective mode of working, as for the former, particularly with regard to the donors, the formal evaluation should defend differences from the initial terms of the contract/proposal if these turn out to have been relevant, reflective and justified.

Rosemary Cairns • I think formal evaluation processes can be helpful in encouraging people themselves to reflect on their peacebuilding capacity. Partly, I suspect, this is by asking questions that encourage peacebuilders and communities to reflect on their achievements in creating and sustaining peace. I worked with a local NGO in eastern DRC over several years and found that part of my contribution was to help them learn terms for what they were doing; give them a language to describe their work; and encourage them to reflect on what made it successful, so that I could learn from them. Last May, the NGO broadcast 3 questions on its weekly broadcast to radio clubs in the region - these were 3 questions I had developed as part of the evaluation. The answers, from local community radio clubs, were given on air (they hold up a mobile phone to the radio station microphone, so the comments can be heard throughout the whole region) and subsequently were translated for me so that I could include them as part of the evaluation report.

Amjad Saleem • i think the question is whether we focus on the system as a means to the end or the end itself. If it is the latter, then often reflective ways of working somehow do not seem to fit in. if it is the former, then we should consider it bearing in mind the caveat of time and being aware of the tensions

Libby and Len Traubman • A determinative question is: Are we intellectualizng "about" peacebuilding, or are we enacting and animating face-to-face relationship building with "the other"? Professors and "experts" in conflict resolution usually are incapable of helping conflicted students engage to transcend their own campus wars. Don't we now need new kinds of labs, even "boot camps", to add the experiential part to the intellectual "study of" peacemaking? An example of African Muslims and Christians engaging like that is in the new 2012 film, DIALOUGE IN NIGERIA -- http://traubman.igc.org/vidnigeria.htm

Alexander Patico • Two somewhat extraneous comments (apologies if they should posted elsewhere): 1) my group (the Orthodox Peace Fellowship) will be have, for its annual conference, the theme of "Conflict Transformation and Peacebuilding" with an area focus on the Middle East. It will be held at the University of the Fraser Valley, Abbotsford, British Columbia, the first few days of June; and 2) inviting everyone to give me feedback on my blog, which concerns US-Iran relations -- found at: www.redhorsedown.blogspot.com -- thanks! 
Alex Patico

 

 

Reflective practice is about releasing the grip of preoccupation with cause-and-effect results.  If interventions take place in uncertain and changing environments, then we must acknowledge that despite the intervention’s intent, the results are out of its causal control.   Reflective practice is about accepting the unexpected and incorporating it into the learning and adaptive efforts of the program.  That said, I wholeheartedly agree with MRogers’ initial thoughts on strengthening reflective practice, especially that of recognizing and labeling the “valuing” done during self-reflection and evaluation.  It is this “valuing”  and its recognition that drive a truly authentic and overall effective intervention process. 

I appreciate this discussion regarding reflective practice and the insight that reflective practice relates to learning, which should be a key component of peacebuilding practices.  A question comes to mind about the type of reflection you might do: simple or critical.

I have recently read some relevant concepts in “Learning Through Reflection” which is a chapter in The Handbook of Conflict Resolution: Theory and Practice.   In it, the authors Marsick and Sauquet propose a model for learning from reflection.  Core to this model is distinguishing between two types of learning: single-loop learning and double-loop learning that stem from simple and critical reflection respectively. [1]

The authors state that simple reflection makes one aware of thoughts, feelings and actions without questioning the interpretation and underlying assumptions whereas critical reflection go deeper to probe those underlying assumptions which impacts upon judgments that have been made. [2] Single-loop learning will therefore come from “simple” reflection on tactics and strategies that need to change to achieve the stated goals based on the static assumptions and values.  I believe this is where the majority of the evaluation systems and reflections take place.  However, double-loop learning comes from deeper, critical reflection and questions the assumptions, beliefs and goals upon which the program is based.  This can be more powerful and substantial difference.  For example, this difference means that you are looking at evaluation data and reflecting not only on “what is the gap here?” (single-loop) but also being open to questions like, “what else is going on in the environment that I may not have been aware of before?” and “ could there be something wrong with my assumptions?” (double-loop) [3] While double-loop learning may be the most valuable type of questions to be reflecting on, you also need to be prepared for the result of this line of reflection may mean to your project.  Questioning assumptions in this way may yield much more substantive changes to the project which comes with additional time and resource considerations. 

You asked the question about how to incorporate reflection and evaluation together.  I would suggest that we should be clear both with peacebuilding projects and evaluation systems which type of reflection and learning are we open to in the project – single or double-loop learning; simple or critical reflection.  In other words, are we reflecting on ways to change tactics to achieve the goals or are we open to reflecting on changing our assumptions and goals.

David

--------------------------------------------------

[1] Victoria J. Marsick and Alfonso Sauquet, “Chapter 19: Learning Through Reflection,” in The Handbook of Conflict Resolution: Theory and Practice, ed. Morton Deutsch and Peter T. Coleman (San Francisco: Jossey-Bass Publishers, 2000), 386.

[2] Ibid 388-89.

 

[3] Ibid 389.

Reflective practice and peacebuilding is a fascinating topic. A common visual image of reflective practice is journaling, but reflective practice goes beyond journaling, and everyone has their specific reflection methods. Personally I feel constrained by writing, and I much prefer going for a walk, and recording my thoughts. Perhaps because my mind works faster than my typing ability, and I become frustrated and I lose my thought process. Perhaps because nature always helps me reflect deeper than sitting in my office. I know my creativity will be renewed by going for a walk, and reflecting on the project. Reflective practice is not a passive process, but very active, and is very useful in the evaluation process whether the purpose is personal or professional.

 

Reflective practice and evaluations is in itself an evaluation tool. It is difficult for the average person to use this tool skillfully without practice in their personal lives. Reflective practice does not produce a measurable, tangible result which makes funders happy. Therefore, it is best when combined with a quantitative evaluation method. It is important as a researcher to have evidence to back up a theory. The same is true with reflective practice and evaluation. There must be sound evidence, only then can reflective practice and evaluation be incorporated.

Thank you for this insightful post MRogers! I tend to think that the challenge of rescuing reflection and evaluation from becoming “accidental adversaries” can be framed into the question of how we design a peacebuilding initiative that can draw from healthy collaboration between the two processes. I agree with you that the hurdles to incorporating reflection in peacebuilding projects come from the challenges such as limited time, a mental orientation for big results rather than incremental processes and nuances, peace practitioners’ fixation with a given project paradigm and yes, donors preference for external evaluation.

As you have identified that “Learning about how peacebuilding works inevitably requires, experimentation, reflection and evaluation,” points to the need for basic paradigm shift in terms of framing an initiative not as a “peace-building project” but as a “peace-building system”. I would add the processes of perception and insight to comprehend a peace-building system as a dynamic ongoing cycle of perception- insight- experimentation-evaluation- and reflection to respond to the conflict situation in a given area. A metaphor for this can be drawn from the way humans adapt to their constantly changing external environments as they make use of similar processes to identify and respond through actions that will have adjustment value; and by improving the quality of their responses or by changing their entire response strategy if adjustment value of an action was poor.

A peacebuilding project conceived of as a peacebuilding system, rather than subscribing to the idea of implementing predetermined activities, may be founded upon the prime objective of putting together structures that will act together as organs giving life to the entire peacebuilding system- more like creating a living entity capable of constantly perceiving and responding to external environment as well as improvising and/or totally changing its response with changes in the context 

Put in the language of project management, the main objectives of implementing a peacebuilding system can be taken to draw from setting up organizational structures that will embody the ongoing functions of perception-reflection -insight-experimentation (project implementation) and evaluation. 

Extending this line of thinking, a twin system of evaluation, internal or external, can be used to determine the overall effectiveness of a peacebuilding system in terms of: 1) How healthy different organs of the system are to perform their requisite functions, 2) how effective the system has been in responding to the conflict by way of conception, design and implementation of incremental actions that have value in terms of their capability towards increasing the overall peace within a conflict area. 

 

The need for such paradigm shift may be discussed in forums like DME and communicated to donors.

I think that this discussion is very interestin. I am a student curerntly earning about evaluation and monitoring methods for a Masters degree. One of the big things I have been learning about is the use and effective applicaiton of reflective practice in evaluations. I really like Jonathon's first post on how reflective evaluation is still very much alive and can help enrich evaluations. It is through my current learning that a reflective practitioner tends to be able learn more about projects. It was my understanding that reflections can help make formative evaluations and allow practitioners to shift the program or paradigm to help address unseen prior needs and make the oerall program more effective. To me, it seems that some form of reflection from the practitioners is almost needed to fully understand the impact of a program. Without some form of reflection how can an implementation evaluation ever present a clear picture of how effective the implementation was? Without reflection how can the practiotioner evaluate how they implemented the program, the challenges they faced, or external forces faced.

 

Im curious to know about other challenges to utilizing reflection and evaluation at the same time? I understand the problems that arise with timing of reflections in relation to the program completion and the safety of participants and evaluatiors in make formed evaluations of a conflict. However, if the right planning occurs, as mentioned before, these problems can be overcome. But what other type of problems are there? How are they being solved or are there still gaps that need be solved? Just curious as a new student to the field.