Congratulations! You have been awarded a contract to evaluate a high-visibility project. You are very excited about getting your feet wet in this world of evaluation!
However, the contract is to evaluate a project in Afghanistan, where there is an active insurgency. The Terms of Reference and the project documents seem to indicate that you will face two main constraints, namely time and the collection of adequate data. You know from your reading that security restrictions will severely limit your movements, and this project is no exception. Despite nationwide implementation, managers want to limit your evaluation to Kabul, which has been in the news in recent weeks due to a series of car bombs and a spike in kidnappings.
On a personal note, you also question:
- Should I increase my life insurance plan?
- What about adding kidnapping, extortion, and hijacking insurance?
- What about medical evacuation insurance?
- Do I contact the embassy beforehand or when I arrive?
- Do I need a “run-bag” for immediate evacuation?
- What do I bring or not bring with me?
In addition to contracting you, the organization has teamed you up with national consultants whom you have never met nor worked with before. From what you have heard through others working in Afghanistan, the qualifications, experience, impartiality, and even the utility of your assigned evaluation team members could be questionable at best.
Without much help from the organization, you design a well thought out, but tightly scheduled work plan to maximize exposure and reach key components of the project. Due to time constraints, it is focused on meeting key project partners and higher-level beneficiaries. You know that your data collection is dependent on the security environment: If a lockdown status were to occur, you would be at risk of not collecting the needed data.
Now, with the preparations complete, you have already fallen into five pitfalls common in high security environments such as Afghanistan!
- The first pitfall is limiting the scope of data collection. Be aware of the tendency for evaluations to take the path of least resistance, which will likely limit the data you really need for an accurate and thorough analysis. In the case of Afghanistan, that translates to project evaluations being Kabul-centric.
- Second is the appropriate use of national consultants. Given this is an insecure environment, you will need to recognize the additional risks national consultants face. At the same time, national consultants admittedly have a much higher degree of freedom of movement. This can be most useful to address your data collection needs if employed correctly, so don’t make the mistake of misusing, underutilizing, or dismissing potentially strong data collection resources. Also, request female consultants to join your team as a means to overcome gender barriers in your data collection efforts.
- Third, too many evaluators see their limited time on the ground and do not include additional data collection measures beyond what they can conduct directly. To strengthen the quality of your evaluation, plan for a mixed method approach in your technical proposal by including perception surveys, capacity assessments, beneficiary questionnaires, or other quantifiable data to strengthen your evaluation. Despite the high security environment, there are many qualified national firms and nongovernmental organizations available to conduct assessments and surveys in tandem to your efforts and ensure that women’s voices are heard as well.
- Fourth, as much as possible an evaluator should not just look at the project from the confines of Kabul, but to get a 360-degree view from outside as well. Security restricts movements to the regional centers, but there is probably room in the budget for the project to send field staff and key informants to regional centers from the peripheral areas where your access is denied.
- Fifth, within a project there will be a focal point colleague assigned to the task of supporting you, and this is usually the individual most familiar with monitoring and evaluation. Nevertheless, it is important to gauge the colleague’s level of knowledge on monitoring and evaluation in general. You should expect that you will be doing some capacity building in order for your colleague to understand what support you need and why. As a matter of detail, you will likely need to establish boundaries with your colleague on their role as a facilitator; it is important they know to excuse themselves from your activities and meetings so as not to skew your results with their presence.
As for conducting the evaluation — RELAX, DON’T PANIC; and as they do in Afghanistan: DRINK TEA! Don’t be overcome by the fact you are entering a high security environment. Knowing these five common pitfalls can help you to avoid and even alleviate them. For the best results, plan well ahead with the organization to incorporate a mixed method approach that will enable the collection of valid, reliable data that allows for future evidence-based decision-making.
About the Author
Mitch Teberg is an M&E Specialist who served more than four years with the United Nations Development Programme (UNDP) Afghanistan in a variety of posts. In his most recent position as the M&E Specialist to the Country Office, he developed an innovative Results Monitoring System that linked project outputs in six programmatic areas (governance, rule of law, livelihoods, gender, environment and health) to the country-specific and the global Strategic Plan of UNDP; strengthened existing M&E systems for improved accountability and aid effectiveness; and led, facilitated or technically supported the development of several large scale projects, a joint-UN agency program, and a Tajikistan / Afghanistan cross-border, multi-agency project, which generated a combined USD 350m in new programming.
Prior to joining UNDP in 2012 he was a consultant and trainer for international and local NGOs in South and SE Asia from 2006. He also worked extensively with Fair Trade artisans, weavers, farmers, and craftswomen and men, as well as the Fair Trade Organizations, NGOs and Community-Based Organizations that support them. Much of his work utilized qualitative data collection methodologies such as key informant interviews, focus group discussions and direct observation, and can be found throughout his blog.
Mr. Teberg earned a Master’s Degree in Sustainable Development from the SIT Graduate Institute in Brattleboro, VT with a focus on Women’s Rights in Sri Lanka; and a Graduate Certificate in Project Monitoring and Evaluation from American University, Washington, D.C. He earned his BA in Asian Studies and Psychology from the University of New Mexico in 2001, graduated Magna cum Laude from the UNM Honors Program, and is a member of the Phi Beta Kappa Honors Society. Upon graduation, he joined the Japan Exchange Teaching (JET) Programme 2001-2003. Prior to his university studies, he served in the US Marine Corps, 1990-1994.
To learn more about American University’s online MS in Measurement & Evaluation or online Graduate Certificate in Project Monitoring & Evaluation, request more information or call us toll free at 855-725-7614.