Good governance requires that citizens monitor power-holders and that power-holders evaluate whether their systems and practices produce the intended positive impact on citizens’ lives. In order for our work on governance and accountability to have legitimacy, we must first demonstrate good governance and accountability within our organization’s own programs and operations. Diligent and carefully planned monitoring, evaluation, and learning are key to showing this accountability to the communities in which we work and our donors.
Monitoring & Evaluation (M&E) is also particularly challenging within the governance field. It takes place in the midst of complicated politics, where quantitative methods of measurement, such as randomized control trials, are almost impossible. It works towards long-term, systemic change that is difficult to measure and even more difficult to do so within donors’ timelines. For example, Accountability Lab’s vision is “to catalyze a new generation of active citizens and responsible leaders.” We are striving to create mindset shifts, and it will take decades for the full results to appear.
Nevertheless, a governance-focused organization can achieve a degree of rigor in its M&E by integrating a process of continual monitoring, evaluating, and learning into its work, rather than just conducting an after-the-fact evaluation. While you cannot expect immediate results at the beginning, it is the most critical time to evaluate what is and is not working, as the costs of change are much greater later down the line. This process of continual learning and adaptation helps organizations keep up with volatile political environments and shifts in public perception.
Just as political will is a critical prerequisite to improving governance, buy-in from key stakeholders is critical ingredient to successful M&E. In order to truly integrate M&E into a program you must:
- Ensure that your entire team understands its value and how it fits into their daily responsibilities.
- Ask the participants in your program for feedback on what indicators are meaningful to them and what data collection methods work best in their context.
Strengthen buy-in even further by being transparent about your objective, creating a safe space for them to give honest feedback, and making sure that none of the terminology that you are using in your survey or interview questions is confusing to them.
One piece of constructive feedback that we received from our participants is that they do not feel confident with their English writing skills and thus filling out long surveys is quite challenging for them. In response, we refined our Impact and Learning Survey to just six key questions, such as “Has the Accountability Lab (or the people it works with) influenced your behavior? (on a scale of 0 to 10)”. We’ve found that the number scale is intuitive to most people, and that it allows us to measure not only the breadth of people we’ve influenced and but also the depth of our influence. We continually adapt the questions based on feedback, while maintaining a few original questions to serve as a baseline.
We also developed monitoring methods that cater to the oral cultures in which we work, including: “Accountability Collectives” in which our local teams facilitate discussions between all our partners in a country about their progress and challenges; Quarterly Impact Calls in which our local partners, participants, and/or experts report on and answer questions about their impact on a public conference call; and video profiles capturing impact stories.
Beyond raising awareness and encouraging responsible behavior among individual citizens and government officials, governance efforts also seek to shift policies and systems. This policy change is a collaborative effort with a wide variety of factors at play, so when conducting M&E on an organization’s influence in the policy arena, one must conduct a thorough contribution analysis in order to prevent any unsubstantiated claims. For example, although the Corruption Perception Index score has improved in both Nepal and Pakistan since we started working there, we have not yet reached a significant enough portion of the population to claim influence on the scores. However, we can specifically track how our Integrity Idol program gave the District Education Officer who won the award enough publicity and prestige that the Ministry of Education reached out to ask for his input on national education curriculum, and numerous other District Education Officers asked for his advice on how to implement similar reforms in their districts.
To us, M&E is about testing the legitimacy of our Theory of Change and continually improving our work. Data is a means to learning, and we have a responsibility to act on what we learn. See Accountability Lab’s Impact and Learning Reports to explore our M&E in more depth.
About the Author
Anne Sophie Ranjbar is Associate Director of Accountability Lab, an NGO that that incubates young people’s innovative ideas for building integrity in Liberia, Mali, Nepal, and Pakistan. She helps lead the organization's strategy, operations, impact and learning, and partnerships, and has also spent 6 months leading its programs on the ground in Nepal. Prior to Accountability Lab, Anne Sophie worked at the Global Fund for Children, KaeMe Foundation in Ghana, The Center for International Humanitarian Cooperation, and the Center for Democracy, Development and the Rule of Law. She received a BA in International Relations from Stanford University.
To learn more about American University’s online MS in Measurement & Evaluation or Graduate Certificate in Project Monitoring & Evaluation, request more information or call us toll free at 855-725-7614.
