One way to gather emic, or insider perspective, definitions of project concepts and indicators is to conduct free list interviews, wherein the evaluator asks respondents to list the terms that they use to define a concept—for example, asking women how they define empowerment. As such, free listing is a way to create an emic definition relatively quickly and it can be a comprehensive definition if we have a good sampling strategy. Together with other research, free listing can go a long way in helping us understand local conceptualizations and pinpoint indicators to measure them.

We oftentimes use free listing towards the beginning of a project. It helps us to define concepts, get an understanding for variances in opinion, and identify key issues.

Depending on research need, evaluators might use random or representative sampling when gathering free lists. No matter what sampling strategy we use, we tend to collect demographic data when we carry out free list exercises. Depending on the project, we might collect data around sex, gender, age, education, or income—it depends on the demographic categories that are important for the project.

Let’s use an example to illustrate how an evaluator might use free listing. Perhaps you are working on a governance program, and as you start the project you free list the term “democracy.” You would intentionally leave the free list exercise open to interpretation, so that people would speak freely about their perceptions. You would simply ask something like, “What is democracy to you?” or “What does democracy mean to you?” Your respondents would list what is important to them in terms of their emic understandings of the topic.

Let’s say you gather 100 free lists of the term “democracy.” Once you gather the free lists, you would need to clean the data. Cleaning refers to creating common term categories, amongst terms with the same meaning. Two terms that might mean the same thing in a free list on democracy are suffrage and voting. You might put these two terms in the same category, so that you stress the importance that these terms play. You would take a leap of faith that these terms mean the same thing, and create one category for them, labeling that category either suffrage, voting, or suffrage and voting. This could mean that this term is pushed up in its frequency of use. The caveat here is that you should not create categories that do not make cultural sense. Of course, you would keep your unclean data in another file, so it is always accessible, in case you want to analyze that version of the data or refer back to it.

After you clean the data, you would analyze it to find early and often-cited terms. To understand frequency, which terms are most important to what people, and which terms can be attributed to what segment of the population, researchers use computer programs for cultural domain analysis.

How respondents order their free lists is very important. Those terms that come to mind first are the ones that we might see as more important in our conceptual definitions. If the term election is coming early in the free lists, then you can assume that this aspect or indicator is central to the research population’s understanding of democracy.

You would also figure out what terms are cited most frequently. You would come up with a list of a few terms that most people used, and likely argue that this is your conceptual definition or cultural domain of democracy amongst this particular group of people. You could also analyze which respondents gave the most common answers. Researchers might use these particular respondents as key informants in future research, as they have the most common perceptions of democracy in the research population.

You would likely then take the particular lists that have these common terms and compare them to the demographic data. This shows us which informants use these terms—and if these terms fall across demographic lines. This is important too—you cannot assume that a particular set of terms is the essence or definition of democracy, for example, if it is only people from one demographic of the informant population is using them.

There are always some outliers in the data. You may do second interviews with these outliers, towards understanding their opinions. This dissenting opinion can be just as important as the cultural domain of democracy we created from the free lists. Evaluators use many tools to understand the emic, without having to spend months living with a population. These tools, and a skill set including cross cultural communications, can go a long way in understanding the emic. Free listing is a useful tool for evaluators looking to define concepts and set indicators to measure both concepts and progress.

References

H. Russell Bernard, “Research Methods in Anthropology: Qualitative and Quantitative Approaches,” 5th edition. Lanham, Maryland: AltaMira, 2001.
Lance Gravlee, “The Uses and Limitations of Free Listing in Ethnographic Research,” Research Methods in Cognitive Anthropology, Spring 2002. http://www.gravlee.org/ang6930/freelists.htm

About the Author

Dr. Beverly Peters has more than twenty years of experience teaching, conducting qualitative research, and managing community development, microcredit, infrastructure, and democratization projects in several countries in Africa. As a consultant, Dr. Peters worked on EU and USAID funded infrastructure, education, and microcredit projects in South Africa and Mozambique. She also conceptualized and developed the proposal for Darfur Peace and Development Organization’s women’s crisis center, a center that provides physical and economic assistance to women survivors of violence in the IDP camps in Darfur. Dr. Peters has a Ph.D. from the University of Pittsburgh. Learn more about Dr. Peters.

To learn more about American University’s online MS in Measurement & Evaluation or online Graduate Certificate in Project Monitoring & Evaluation, request more information or call us toll free at 855-725-7614.