Design, Monitoring and Evaluation for Peacebuilding

You are here

Assessing Relevance

The OECD-DAC determines relevance to be one of the five core criteria for evaluating development assistance and defines relevance as “the extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor.”1 Despite its centrality to the field, measuring relevance continues to be a methodological challenge and more research is needed to investigate this criteria.23 

What is Relevance?

Cheyanne Scharbatke-Church in her article “Evaluating Peacebuilding: Not Yet All it Could Be” points out that the OECD-DAC’s definition of relevance is more specific for the peacebuilding states, “the extent to which the objectives and activities of the interventions(s) respond to the needs of the peacebuilding process.”4 Ms. Scharbatke-Church points out that this minor change in language requires an evaluation to be rooted in a conflict analysis, which compels peacebuilding expertise from an evaluator in that he or she must be accustomed to nuances of conflict frameworks and the difficulties of making connections between analysis and evaluative conclusions.5

Mark Rogers, in his piece for CDA Inc. Evaluating Relevance in Peacebuilding Programs discusses six dimensions of relevance established from criteria-based evaluations such as the OECD-DAC.6 

  • First, like Ms. Scharbatke-Church, he determines that relevance in conflict prevention and peacebuliding programs must be based upon a conflict analysis. In order to address peacebuilding needs, program design and implementation are based upon current, accurate conflict analysis.
  • Next, the programming goals and objectives must align with peacebuilding needs.
  • The third dimension is timeliness, or operations should coincide with advantageous and/or critical moments.  
  • Fourth, relevant programs are adaptive and responsive to changes over time.  
  • Fifth, stakeholders and observers must also perceive the program as relevant.
  • The sixth dimension addresses strategic and policy alignment. Relevant programs and/or interventions align or integrate themselves into thematic priorities (for example health, youth, gender, etc.) and/ or the policies of governments and/or donors. These dimensions are not mutually exclusive and should be situated within the larger framework of the OECD-DAC criteria as there is potential for some overlap.7

Mark Rogers also proposes standards for evaluating relevance of peacebuilding programming.  These can be found on pages 4-5, and are expanded on with potential lines of inquiry that will help evaluators reach conclusions on those standards. 

Any evaluation methodology can include an assessment of relevance.  This includes outcome mapping, theory-based, most significant change, and impact evaluation.  An assessment of relevance involves the inclusion of specific lines of inquiry, which Rogers conveniently offers suggestions for, and is not dependent upon any specific methodology (other than, of course, triangulation!).

Relevance of a project or program cannot be determined without a conflict analysis.  It furthermore requires a certain level of knowledge of peacebuilding concepts, programming and processes in order to make a determination of relevance.  As there are different aspects of relevance, and an evaluation is unlikely to be able to assess them all, terms of reference should clearly identify which aspects of relevance will be evaluated based on the learning needs of the project or program. 

As Rogers wrote, “deconstructing the notion of relevance into different dimensions, while helpful in understanding different components, presents real challenges in offering a consolidating evaluation finding related to relevance.”8 The numerous challenges and their seemingly complexity can be overwhelming.  But, these challenges can be overcome, and distinguishing between that which is simple, complicated and complex in evaluating the specific dimensions of relevance may assist evaluators and project staff in preparing for and conducting the evaluation, drawing conclusions, and making recommendations. 

 

 

This is really interesting, Tiffany. Thanks! It raises some serious questions about the standards by which relevance might be determined - which due to the conextual nature of peacebuilding programming, would seem to be a large task.  

Mark Rogers lays out some broad guidelines for us to consider, but the answers to each of these might vary depending on how different theories of change, strategies, activities, outputs, outcomes are valued by the evaluator.  After all, we might agree on a conflict analysis and the key driving factors, but the entry and leverage points will differ from organization to organization depending on their preferred methods and capacities.  

I wonder if a scaled determination of relevance could be useful here?  It break down each activity or outcome into its own scale, and chart the relevance of that particular work strain.  The medium of each of the scales could be used to determine the overall relevance rating of the project or programme.  I'm not sure if this has been applied, but it's a thought.