From Gears to Gardens: Changing Our Perceptions of M&E System Design for Greater Impact

Patricia J. Rogers
12 September 2022
Image
From Gears to Gardens: Changing Our Perceptions of M&E System Design for Greater Impact
When we talk about a monitoring and evaluation (M&E) system, what sort of system are we thinking about? Do we have a narrow or broad understanding? And how should we design, plan and implement a system that effectively supports monitoring, evaluation and evidence use?

Patricia J. Rogers is an Independent Consultant and Founder/former-CEO of BetterEvaluation.

Writing and editing support provided by Maria Fyodorova, Communications Consultant for GEI.

 

Changing Our Perceptions of M&E System Design for Greater Impact

When we talk about a monitoring and evaluation (M&E) system, what sort of system are we thinking about?  Do we have a narrow or broad understanding?  And how should we design, plan and implement a system that effectively supports monitoring, evaluation and evidence use?

An Information Technology System for M&E . . . or Something More?

From my experience working with people in a variety of government and development organizations, I have seen an “M&E system” defined quite narrowly.  It is often thought of as an information technology (IT) system for managing monitoring data (e.g., through a set of spreadsheets, a database or customized software), including specific requirements for conducting evaluations periodically.  In certain cases, it might be understood a little more broadly to include the organizational policy documents, guidelines, and training required to implement the IT system and manage the evaluations.

However, when the Global Evaluation Initiative (GEI) refers to a “national M&E system” it means a country’s overall “ecosystem” - all the structures, attitudes, incentives, approaches, policies, laws, behaviors, skills and abilities that contribute to whether robust monitoring, evaluation and evidence use happens.  This includes informal processes and organizational culture – including how people are treated when they identify problems, admit mistakes, suggest changes to plans, or seek to learn from successes that have been achieved elsewhere.  How we design, plan and implement an M&E system needs to address this broader understanding of what an M&E system needs to encompass.

Supporting the Different Functions of an M&E System

Another way that an M&E system is often narrowly understood is in terms of its functions.  It is often understood mostly in terms of upwards accountability - checking that processes, plans and targets have been met.

But M&E can and should play a number of different functions:

  • Accountability. While it needs to support upwards accountability to policymakers, senior management, and funders, it also needs to support horizontal accountability to colleagues and partner organizations, and downwards accountability to those who are intended to benefit from programs and policies. 
  • Learning. It needs to support diverse types of learning by staff and others, including appropriate adaptation of learning from different sites and contexts. 
  • Localization. As well as meeting central organizational requirements, an M&E system should provide space and support for localized M&E that meets local needs.

An organizational or national M&E system therefore needs to include:

  • Centralized processes for aggregating and reporting data; but, also localized processes for using that data and other data to inform operational decisions and longer-term policy/investment decisions.
  • Formal procedures and structures; but, also informal processes and organizational cultural norms that support M&E.

 

Limitations of Linear Design Processes

The classic approach to the design and implementation of M&E systems is a linear, top-down approach of, first, developing a statement of requirements, then developing a design for a system, and finally implementing it. 

In computer software development, this “waterfall” approach has been, generally, determined to be a failure. It was determined that the best system design is often not evident until end-users use the system and there is a better understanding of implementation.  What emerged from this failure was the “agile software” approach that focused on quickly developing and launching a minimally viable product, using it to clarify needs and design requirements, and continuing to iteratively improve upon it throughout implementation.

M&E system design and development would benefit from a similar iterative approach, especially given the need to support local decision-making needs and informal processes, which are likely to be always changing.

Shifting From Gears to Gardens

One way to consider the shift discussed above is as a move from thinking about an M&E system more as a machine - where functions are defined in advance and the system is designed in a linear way to achieve those functions - to thinking about M&E system design through a gardening metaphor.

If we think about an M&E system as a garden, then we can support emergent and adaptive responses that suit local conditions.   In a good garden, there is an overall plan, but one which is continuously adapted as new challenges and opportunities arise, informed by ongoing learning about what is needed.  As in a garden, there would be ongoing maintenance (e.g., adding data, tweaking reports) and seasonal tasks (e.g., periodic reporting, and cycles of reflection and revision – perhaps some quarterly or monthly, and others annually). 

Instead of seeing central agencies’ role as being primarily about planning and compliance control, it might be more useful to think about them as stewards of the various components that contribute to a healthy national M&E garden. Like in a garden, there will be a need to plant, water, prune and weed (i.e., set up processes, support them with ongoing resources, trim and shape them appropriately, and remove what doesn’t add value and that takes away resources from valued activities). 

To improve M&E system design, we must move away from viewing it as a linear process. Just like a successful garden, an effective M&E system must be able to flourish in an always changing environment with interdependent and complex components – something a machine just can’t do.