Gregory Warner
The coursework for American University’s Certificate in Project Monitoring and Evaluation (PME) and Masters in Measurement and Evaluation (MME) offers students practical experiences in evaluation. The Certificate and Masters programs provide several opportunities for students to partner with organizations and carry out research and evaluations through various practicum assignments. For example, in Principles and Theories of Evaluation, students work with a partner organization to construct a logic model and Evaluation Statement of Work; in the Qualitative and Quantitative Methods courses, students partner with organizations to design projects and carry out research; and in the Approaches and Design Course, students partner with an organization to carry out an evaluation. Evaluator faculty mentor students throughout these processes.
Getting practical experience in evaluation as part of my studies is what drew me to American University’s Certificate in Project Monitoring and Evaluation program. As one might imagine, designing, researching, and writing an evaluation report was an incredibly gratifying experience. It provided me an opportunity to apply my PME knowledge in a real-world evaluative environment, and to be mentored through the challenges by one of the program’s faculty members.
For my final project, I partnered with The Red Pencil, a non-profit organization that uses art therapies “to enhance the mental and emotional health and wellbeing of those affected by traumatic circumstances and life atrocities.” For the project, I designed an evaluation, conducted research to support it, and wrote an evaluation report. I did all of my work virtually.
As I reflect on my studies in the Program, I hope to highlight and pass on three key lessons learned to current and future students, and to those writing their first evaluation reports.
Lesson 1: Access is Key
As you start your PME coursework, you will learn that the program requires you to partner with organizations to carry out evaluation related research. Sometimes students partner with their employer or an organization where they volunteer to identify research and evaluation needs. This can help to streamline coursework and assignments so that they build off each other. Many students forge new relationships and partner with nongovernmental organizations in their local communities for their coursework.
I found that partnering with local organizations like The Red Pencil gave me experience forging new relationships and navigating unknown contexts. Through the experience, I also developed good interpersonal communications skills. These are all really important skills to have as an evaluator. I was responsible to consult with the organization, learn about its theory of change, craft their logic model, set the design of the evaluation, carry out the evaluation research, and write an evaluation report.
As you approach your studies, I would suggest working with an organization and your instructor early in the process to secure placement with a practicum partner. I found that many professionals from nongovernmental organizations were familiar with American University, and this helped as I networked for practicum partners. My professors were also helpful in identifying potential partner organizations.
Try to partner with an organization that is open to the evaluation and the evaluation process, and will give you access to existing data and interview respondents. In addition to your faculty mentor, it is helpful to have a key informant at your partner organization that can help introduce you to others in the organization.
Lesson 2: Start Early, Stay Flexible
The coursework in the program goes quickly, and students should be ready to start their practical work early in the semester. As soon as you have a partnership agreement, I recommend starting your work! It takes time to learn about an organization, design and carry out your research, and write a report.
When you are conducting an evaluation and writing an evaluation report, I would recommend meeting with project stakeholders to learn about the organization and the context in which it operates. Confirm if the organization has a program theory of change, and if not, let this be the start of your discussions with your partner. As you outline the theory of change, consider a logic model for the organization. Discuss the purpose of the evaluation with key stakeholders, and let this guide the design of your evaluation, your methodology, and your report.
As you collect data, you should expect to see certain trends emerge. Sometimes it is difficult, but do not draw early conclusions that could taint the collection of future data! It will be tempting to shift the report to focus on particular outcomes as they emerge, particularly if they are positive, at the expense of ignoring what could be important data.
It is also important to stay flexible, and keep in mind that the research and report are very much living documents. For example, my original methodology for my evaluation included questionnaires. After engaging in some initial research, I discovered that questionnaires were not well-suited to answer my evaluation questions. After consulting with The Red Pencil, we collectively determined that semi-structured interviews (SSIs) with key informants would be the best instrument to collect the desired data. As you can imagine, SSIs can take very long to conduct and to transcribe! In my case, most of my interviews took place in time zones which were eight to thirteen hours ahead of Eastern Time, and required additional flexibility on my part.
Lesson 3: Work-Life Balance and Your Graduate Studies
The Capstone report is inclusive and collaborative in nature; you need to remain flexible to meet the needs of your partner organization, but systematic so that you can stay organized and on task. The collection of data is time-sensitive and your respondents will have their own schedules, which you have to accommodate. If you are a full-time working professional, you may find that balancing work and life becomes challenging. While completing the work for my evaluation report, I had to carve out time to accommodate research and writing.
One of the best decisions I made was to discuss the report with my employer, family, and friends. After I explained my studies and practicum assignments, they all understood that I was essentially a one-person evaluation team! After this, they were in a better place to support me in my endeavors. I also kept open communication with my American University instructor, who was very helpful in providing timely feedback throughout this process.
My final advice to you? You will find that your PME or MME coursework will be an experience to remember. Take the opportunity to network with your instructors, fellow students, and potential practicum partners. Use your instructors as mentors, and keep open communication with them as you design and carry out your practicum assignments. This is particularly the case for classes requiring an evaluation report. All of this will enrich your educational journey at American University. Good luck!
BIO:
PME Certificate graduate Gregory Warner currently works as a government civilian contract specialist in Kansas City, Missouri. He holds a Bachelor of Science in Political Science from Old Dominion University and a Master of Art in International Relations from Webster University. Heavily influenced by his eight-year military career, he loves to travel, write and publish poetry, and to discover new culinary experiences. He enjoys writing about world affairs and has a working paper titled, Cold Peace: Non-governmental Organizations and the North Atlantic Treaty Organization: Contributing Forces in the 2014 Ukrainian Crisis, published in the Webster International Relations Review (WIRR).
In March 2016, after completing his Master of Arts in International Relations, he became interested in evaluation in the non-profit sector. This interest was developed during his International Relations studies at Webster University. After conducting some research on the career field and institution that offered the training program, he applied to and was accepted into American University’s (AU) Project Monitoring and Evaluation (PME) Certificate course in May of 2016. He graduated with his PME Certificate in December 2016 and carried out research with The Red Pencil for his Certificate capstone.
