Rapid Ethnography

In David R. Millen’s article, Rapid Ethnography: Time Deepening Strategies for HCI Field Research, we learn about several tools and techniques to perform effective ethnographic studies on a realistic (in corporate terms) time table. As Millen points out, one of the great challenges facing HCI professionals in the corporate world is that product development cycles are short and constantly getting shorter. The key to success for many businesses is being first to market with new products or ideas. For this reason, lengthy user research, testing, or in-field ethnographic studies (which this article focuses on) are generally not possible. While we, as user experience advocates, may view this type of work as an essential component to any product development cycle, senior level executives (the people controlling our budgets) are often not convinced. At many corporations, a focus on product usability during product development may not be an existing part of the product development cycle. At my current place of employment, formal user testing or contextual inquiry (or any other form of usability study) is never done. In order for companies like mine to adopt these methods, or for companies already performing some amount of usability analysis to enhance their toolkit by adding ethnographic study, it is important that this technique be as quick and painless (financially speaking) as possible, and I think Millen has done a good job of showing us how to make that happen.

He begins by providing a brief background of ethnography in general, and a discussion of its value to HCI researchers. He describes an apparent misconception by many in the field of HCI that ethnography is simply a means of gathering data from the field, as opposed to also being a method of analyzing and making use of that data. He tells us that “ethnography is rather a form of analytic reportage, with the ethnographer acting as a translator or cultural broker between the group or culture under study and the reader.” He goes on to explain that while researchers in other fields successfully use ethnography for both data gathering and analysis, they take along time to do it. “Typically,” he says, “ethnography will take place over a period of several months with at least the same amount of time spent in analysis and interpretations of the observations.” As mentioned earlier, that sort of time table makes ethnographic study an impossibility in a modern fast-paced development environment. Millen attempts to remedy the situation by providing us with three basic techniques that can be used to greatly reduce the time required (and increase the effectiveness) of ethnography for HCI purposes.

First, Millen suggests that we should “narrow the focus” of our study. In traditional ethnography, it is common to collect as much data as possible about the subject being studied, and to determine which of that data is useful during the analysis phase. By deciding what types of tasks/activities we want to observe, or what kind of data we need to collect before observation actually takes place, we can the time wasting associated with the collection of useless data. After determining what data is sought through an ethnographic study, Millen recommends the use of strategically chosen “informants,” to guide the observation. He tells us that using various types of informants (field guides, liminal informants, and corporate informants) will allow us to more quickly locate and observe the desired behavior/activity. Field guides “should be people with access to a broad range of people and activities and be able to discuss in advance where interesting behaviors are most likely to be observed…” Liminal informants are “fringe” members of a group, meaning that they interact with them in some indirect way. Millen says that these fringe members can often “provide …an interpretation of the events based on prior events or experience” with the group. Finally, corporate informants are members of the research team who may normally work in other functional areas (like marketing, as opposed to HCI). Corporate informants, though often under-used, can provide valuable new perspectives on the activities behavior being observed during ethnographic studies.

The second technique that Millen suggests is interactive observation. He mentions a couple of interaction observation methods, the first being “interactive feature conceptualization.” Millen doesn’t seem to explain what interactive feature conceptualization actually entails, which is disappointing. I wasn’t able to locate a free copy of the article he references with regard to this term, and I couldn’t find much information elsewhere in scholarly work, so I’d be very interested to know exactly what he means here. The other interactive methods he alludes to are structured interviews, activity walk-throughs, contextual inquiry, the Group Elicitation Method, and participant observation. I am personally fond of the last method mentioned (participant observation), which Millen describes as the researcher actually participating in the activity of interest. A few years ago, I wrote product documentation for Dell Inc, and one of the things that I miss most about Dell is the fact that the Technical Publications group had its own lab, where we could go to experience any of the procedures that we were documenting. A full-time lab staff was always available to set the lab up as necessary, so if I needed to test the installation procedure for a particular wireless router on a particular desktop system, running a particular operating system, I would call the lab, tell them exactly what setup I needed, and then I would go perform the installation of the router myself. That way, I would experience the same obstacles, confusion, and frustration that a user might experience, and I could modify my documentation to avoid those situations. I rarely have the opportunity to test products myself at my current job (we make drills and saws for neurosurgery), which certainly hinders my ability to create usable documentation. I think Millen’s “participant observation” is an excellent (and certainly efficient) way to see things through the eyes of the user.

The final technique described by Millen is collaborative data analysis. At one point Millen recommends having more than one researcher in the field, saying that while, “there is always the chance that the presence of more than one researcher will be significantly more disruptive to the ‘natural’ setting, there are several advantages.” He continues this theme when he tells us that the analysis phase should also be a collaborative effort, in order to avoid a “painstakingly slow” process. I’m not sure if Millen was referring to collaboration between researchers during the analysis phase, or collaboration between researchers and technological tools that they might use for analysis (which he goes on to discuss). I would assume that data analysis is generally a collaborative effort, so this doesn’t seem like a novel idea, but I could be wrong, as I’ve never worked in HCI or really performed ethnographic studies in any other field. When discussing collaborative data analysis, Millen tells us that many tools are available to ease the burden of making sense of the mass quantities of data collected. He points to computer applications designed to analyze data in a variety of formats, including text, images, and video. He also recommends the use of manual analysis processes designed to give researchers a better understanding of the activity they observed. He points to cognitive mapping, pictorial story telling, and scenario analysis as effective ways of analyzing what was observed. He describes a specific type of cognitive mapping called causal modeling, which basically consists of drawing a diagram depicting how various actors or objects in the activity interact with one another. This seemed like a very natural and effective way to understand new or unfamiliar activities. I often use this method in my own work when trying to understand how a new system or process works or is organized.

Millen goes on to describe a case study, which illustrates the method he has described. I think that Millen’s approach to ethnography is a sensible one, and one that has the potential to make ethnographic study a more viable possibility in modern product development. Despite tightening budgets, and compressed development schedules, I think that the great benefits that can be derived from a true understanding of a product’s user base, combined with the minimal cost (in both time and financial expense) of performing ethnographic study rapidly (as described here), bring ethnography out of the realm of impractical academic theories, and into the realm of truly useful development tools.

References

Miller, D. Proceedings of the Conference on Designing Interactive Systems. ACM, 2000.

Bauersfeld, K. & Halgren, S. “’You’ve got three Days!’ Case Studies in Field techniques for the Time-Challenged.” In D. Wixon & J. Ramey, Eds. Field Methods Casebook for Software Design. John Wiley & Sons, 1996.