American University’s (AU) Measurement & Evaluation (M&E) program blends quantitative and qualitative methods, evaluation theories, intercultural communication, and ethics that serve students across multiple sectors. This blending works because the human experience is critical to providing context and increasing the value of data.
Many expect the demand for data analysis and related professional skills to grow over the next 20 years, as more businesses and government agencies respond to “data democratization,” or the increased access to and volume of data available.1 Evaluation professionals are data influencers, often guiding what data are captured and analyzed, methods used, and data sharing. Data inform strategic business decision-making, ensure governmental accountability, assist community planning, and measure program effectiveness across diverse settings. Indigenous nations and tribally owned enterprises are rethinking how they use data too, increasingly relying on e-Commerce and big data analytics to drive reservation economic development while exercising data sovereignty.2
Evaluators’ quest for data must be conducted ethically and meet professional standards. This is crucial when conducting evaluations within historically/currently underrepresented or racially minoritized groups – such as Indigenous nations, communities of color, LGBTQ+, and Two Spirit peoples – often observed by “outside” evaluators using deficit-based assessment models.3 I partnered with a First Nations Tribal Council to consider how evaluation could support their self-governance goals for my M&E capstone project. Before developing an evaluation plan, I needed to learn the historical, cultural, environmental, political, and economic contexts wherein their evaluation would happen. This learning included research on treaties, geographies, legal/financial complications created by Canada’s Indian Act, ongoing language and cultural revitalization efforts, and the roles of elders and traditional knowledge keepers in the First Nation’s governance system. I talked to several advisors who had previously worked with the Council on related projects and could provide insights into communication protocols. These initial steps went a long way towards building trust and demonstrating my respect for the community as a non-Indigenous evaluation student before “the work” began.
While generally relegated to the social justice “branch” of evaluation theory, culturally responsive and equitable evaluation approaches (CREE) are becoming mainstream as more foundations and institutions commit to improved diversity and inclusion practices. Though “disruptive” information technologies like artificial intelligence and demands for quantitative data grow exponentially, there remains an equally compelling drive to center the human experience often best captured through qualitative, community-centered methods. For example, qualitative methods may better reflect an Indigenous community’s practice of storytelling and oral history, provided that evaluators respect and protect Indigenous knowledge.4
The American Evaluation Association (AEA), Canadian Evaluation Society (CES), the Center for Culturally Responsive Evaluation and Assessment (CREA) and other professional evaluation associations offer guiding standards and resources. Further, association-sponsored conferences, workshops, and publications provide excellent continuing education; an AEA Indigenous Evaluation workshop served as a touchstone for my capstone project.5 AU’s M&E program directly aligns with AEA standards and the evaluation field’s best practices. Students balance data-driven and human-driven perspectives, especially through service-learning practicums which set the program apart from other data measurement or evaluation studies. In M&E, students apply course concepts to real-life situations, developing personal skills as they partner with local organizations; for me this offered multiple opportunities to cultivate “trusted advisor”qualities, which I continue to use as a professional consultant. Service learning is particularly valuable in Indigenous studies, as it allows students to listen, learn, and share knowledges that deepen cultural competency, honor Indigenous worldviews, and benefit tribal communities.6 My practicum and capstone experiences demonstrated how vital personal connections, respectful communication, and reciprocal relationships are for any evaluation setting.
About the Author
Michael Petillo, MS (he/they) is a Principal Consultant at CES Partnership, LLC. Michael’s background in nonprofit program management, HIV/AIDS service, LGBTQ+ advocacy, and international community development informs his focus on community participatory, culturally responsive and equitable evaluation. He earned a Master of Science in Measurement and Evaluation at American University in May 2020, and completed a Graduate Certificate in Tribal and Indigenous Nation Building at Northern Arizona University in May 2021. His recent academic work focuses on how culturally responsive evaluation can support Indigenous nation self-determination and sovereignty.
_____________________________________________________________________________________
1. Purdue University Global (January 15, 2021) Rise of the Data Analyst—What’s Behind the Boom? [blog]. https://www.purdueglobal.edu/blog/information-technology/rise-of-data-analyst/; Marr, B. (July 24, 2017). What Is Data Democratization? A Super Simple Explanation And The Key Pros And Cons. Forbes. https://www.forbes.com/sites/bernardmarr/2017/07/24/what-is-data-democratization-a-super-simple-explanation-and-the-key-pros-and-cons/?sh=827d8966013f
2. Rainie, S., Rodriguez-Lonebear, D., & Martinez, A. (2017). Policy Brief: Indigenous Data Sovereignty in the United States. Native Nations Institute, University of Arizona. https://nni.arizona.edu/application/files/1715/1579/8037/Policy_Brief_Indigenous_Data_Sovereignty_in_the_United_States.pdf
3. Tuhiwai Smith, L. (1999). Decolonizing Methodologies: Research and Indigenous Peoples. Zed Books, Ltd.: London.
4. Smith, ibid.
5. LaFrance, J. (November 13, 2019). Foundations of Indigenous Evaluation [presentation]. American Evaluation Association Conference.
6. University of Sydney (January 19, 2021). Service Learning in Indigenous Communities [webpage]. https://www.sydney.edu.au/students/service-learning-in-indigenous-communities.html
