How Soccer Can Help Us Be Strategic About Strengthening Monitoring and Evaluation Systems

Gonzalo Hernández Licona is the Director of the Multidimensional Poverty Peer Network and a frequent consultant to multilateral organizations like the World Bank Group and the UN.

Hernández Licona founded the National Council for the Evaluation of Social Development Policy (CONEVAL), an independent public institution, whose main objective is to evaluate social programs and measure poverty in Mexico. Prior to leading CONEVAL, Hernández Licona served as the General Director of Evaluation and Social Program Monitoring for Mexico's social development ministry (SEDESOL). 

He was one of 15 experts in charge of developing the 2019 Global Sustainable Development Report. He holds a PhD in Economics, University of Oxford.

Editing support provided by Maria Fyodorova, Communications Consultant for GEI.

Pregame Analysis

Geir Jordet is a professor at the Norwegian School of Sport Sciences where he conducts research and teaches psychology and soccer. His main research is explaining the psychological reasons why a player can perform better on the field. Two of his conclusions are very applicable to efforts on strengthening monitoring, evaluation and evidence use in countries. 

The first conclusion is that the player who always scans the field plays better because they are clear about where all the players are and what their main routes and strategies could be. A player who only looks at the ball when they receive it, and only looks at the opponent who is in front of them, loses valuable information about what is happening on the rest of the field. The second conclusion is that when a player analyzes, beforehand, how the goalkeeper and the opponent team behave during a penalty shooting, the shot ends in a goal more often than if the player takes the penalty without doing this analysis. In both cases, player performance improves when time is taken to analyze the situation. 

If diagnostics are important to address the complexity in sports competitions, then they are certainly crucial in public policy – including the development of monitoring and evaluation (M&E) systems, a key need for any government interested in improving public policy and bettering the lives of its citizens. M&E systems allow governments to gather and use data to determine what is working and what needs to be improved. Over the longer term, M&E can improve public institutions by creating transparency and accountability.

For more than three decades now, developing countries have been increasingly interested in building M&E systems and improving their national evaluation capacities (NECs). Chile, South Africa, Benin, Colombia, Uganda, Mexico, and Ghana are all examples of countries that have improved their NECs in recent decades. In most cases, the process to improvement capitalized on initial diagnostics around existing M&E frameworks, processes, practices and capacities.

For example, when Mexico decided to pursue a new evaluation architecture in 2006, both Congress and the federal government, with some help from the World Bank, analyzed the advantages and challenges associated with M&E during the previous period. There had been an important increase of evaluations since 2001 due to a mandate from Congress, but the quality of the evaluations was relatively poor. The evaluation of federal programs usually lacked independence and quality because the external evaluation teams were hired directly by each program manager. At the same time, most programs only reported financial and outcome indicators. This diagnosis led to an improvement of the M&E system in 2007, with the creation of a new independent evaluation agency; a mandate for this agency to coordinate with the Ministry of Finance; the development of results indicators in all federal programs; and, the establishment of quality assurance systems. 

The Pitfalls of Skipping the Diagnostic

Despite the importance of having a good pregame analysis both in sports and in public policy, sometimes it is tempting to skip the diagnostic phase. There are a variety of reasons that I have come across in my many years of work with national M&E systems.

1. There is an urgency to perform quickly. Public policy often moves at the heightened pace of the local political landscape. Governments are not eternal - especially when they are democratic. They want to generate quick results and capitalize on them politically. This rush can make it seem like a diagnostic process might be a waste of time and delay implementation and results. Yet a badly implemented program or national reform can have lasting political impacts.
2. Problems are problematic. A diagnosis shows what is working - but it can also highlight challenges. If I see my missed penalty shots on video, even with the intention to improve, I might feel ashamed about the mistakes (since it is human nature to magnify mistakes when they are ours). However, leadership often involves admitting that some things might have gone better. Diagnostics help you see how to improve in a strategic way.
3. Perception that the problems are already known.  Personal experience or anecdotal information often informs the decisions of policymakers.  But without a structured investigation, it is difficult to ascertain if all the right questions have been asked or all the details examined.
4. Need to save resources. This is perhaps the most common reason. But investing in analysis before investing in implementation – to determine the most strategic approach – is always the most efficient use of resources.

Fortunately, governments, national and international institutions increasingly have invested in good M&E diagnostic tools in recent years. The Global Evaluation Initiative (GEI) has developed a Monitoring and Evaluation Systems Analysis tool (MESA), with the aim of supporting a government with producing a comprehensive diagnosis of a country’s M&E systems. The MESA is like scanning the field to see what is happening everywhere.  It looks at the economic, political, and social environment (including attitudes and behaviors around M&E) - but can be adapted to local contexts, needs and interests. A MESA is not an end in itself, but rather a means to gather, structure, and analyze information to inform and shape improvements to a country’s M&E systems. If you are about to develop or improve a national M&E system, we encourage you to apply a MESA. The chances of taking the right steps in a complex M&E system will increase with such a diagnosis. If, on the other hand, you are about to shoot a penalty in a Sunday morning match, invited by your child, spend time and concentrate before kicking the ball, so you will increase the chances of scoring and making your kid happy!
_____________________________________________

Please join us! Comment on this blog or reach out to us on social media: LinkedIn and Twitter.  What has been your experience with diagnostics for monitoring and evaluation systems? Share your stories with us.  

You can sign up for our newsletter here.

If you would like to contribute your knowledge to this blog, we would be happy to work with you - please contact us at contactgei@globalevaluationinitiative.org.