Users, Audiences, and Design

Reading Response - 9.16.08 ~ Matt Rolph

Doyle's Sherlock Holmes could deduce a variety of things -- like a person's profession, where he or she'd been recently, even his or her unspoken secrets -- by a variety of cues. It is an exaggeration, of course, and reflects a variety of Victorian clichés and prejudices (do criminals have a particular ear shape?), but it remains plausible to readers because of a basis in truth: even as we use our tools to shape our surroundings, our tools shape us. For HCI, there are some implications -- given the data, we may spot, for example, a Mac user using a borrowed PC. And, if we are effective designers, we use the foresight granted by reason and experience to envision the different ways different users may use our designs.

DILLON & WATSON

Dillon and Watson report that "in current situations this often means distinguishing users broadly in terms of expertise with technology, task experience, educational background, linguistic ability, gender and age." There are other models, and they cite Nielsen's and Booth's examples, before revealing their purpose: to consider and “psychology of individual differences" as it applies to design and user training.

We get an abbreviated history of psychometrics and definitions of various approaches. In theory, I'm very interested, but these are the sorts of tests and measurements I tend to question.

Differential psychology sets out to look for and quantify difference in ways that are seldom positive for everyone in the sample, and generally presumes that something like intelligence is constant -- that a measurement taken today (using a questionable instrument) will have some value tomorrow. Consider the list on page 6, and imagine we are measuring your broad cognitive speed today (surprise! It'll be fun, except for the part later where we divide top scorers from the others and give them all the ice cream). Are you ready? Going to enjoy getting this number? Think it might be better if you'd slept more? That's not part of the theory ... here, wear this pointy 'I am special' hat. Incidentally, these authors spend a lot of time saying things like "this ... has been highly praised by many reviewers" -- I want to know their names, and disagree that this is not a contentious issue. The number of factors and the way these are organized is not the primary distinction between theories except in a very narrow field of inquiry.

Similar criticisms of the other sections are possible. Cognitive psychology, which "originally tends to be more interested in general models of processes than in individual differences”, is a little less noxious on the surface. It is here, however, greatly simplified and doesn't take into account any of the new data from new tools that allow the measurement and visual representation of neurological activity during cognitive tasks. These authors are focused on identifying some key terms and correlating them to terms in HCI, i.e. latency and speed. Personality and cognitive style are interesting concepts, but as easy to misuse as they are to use -- imagine if your computer operating system tried to 'help' you based on what it perceived as your neuroticism or extraversion indexes. The result could easily be something like Clippie the paperclip in Office ("it looks like you're writing a letter ..."). The psychology might attract funding from investors, but success in the marketplace would depend on user perception, and many users, despite their need for help, are sensitive to anything like condescension.

Psychomotor differences are relevant, but Thorndike's research (which suggested practice does not make perfect at all) isn't particularly applicable. Yes, some people are better at some things ... so?

As you may guess, I disagree with some of the six "practical implications of 100 years of differential psychology." I wrote out and deleted a line by line rant here. These hinge on the assumption in #1 that "a core number of basic abilities have been reliably and validly identified," something I don't believe the evidence presented here supports. I want to see that happen -- I want to see the promise of improvements for all users fulfilled -- but there are some major concerns. A "more rigorous theoretical and data-driven approach" could be good, or it could simply validate various preconceptions about the end user and his or her capacities, counterproductive if the goal is to actually improve user experience for all.

The presumption that low-visual-ability users will need help to perform as well as high-visual-ability users could result in a situation in which the interface is slower and more obnoxious for the very people we are trying to serve better.

"25% of variance on the basis of ability alone" is a made up number with no practical utility whatsoever ... the authors hedge their bets here, saying it shouldn't be the "end of the story." That is correct, gentlemen. It is not.

Though I've become antagonistic of the article, I do agree that analysis of individual difference is useful, that more study is warranted and necessary, and that we can gain a better understanding of the people we design for. I even think psychology could be useful to that end, provided we very carefully consider our established assumptions.

LIGHT & WAKEMAN

All of us recognize the value of the positive individual experience with a website. The trick is to arrive at implementable suggestions likely to improve user experience across the board. The master trick involves documenting the reasons for your suggestions in a manner than demonstrates their validity in context, with real users.

Light and Wakeman provide their methodology, appear more sensitive to user concerns, and their recommendations seem far more logical and supportable. Where Dillon and Watson are sort of saying 'psychology is useful, here are some theories', Light and Wakeman are actually getting somewhere.

I may like their recommendations more because each appears to show the user the deference and respect he or she is due. Respect is a very useful term in this case. I'd suggest that it is possible to attempt to show respect for a wide variety of people without necessarily measuring them against an abstract standard (with or without agreed upon metrics).

MENTAL MODELS (PAYNE) AND CONCLUSIONS

Let's think about Sherlock Holmes a little more. His deductions are more accurate than is likely in part because of the structure and content of his society, a simplified representation of Victorian England, and in part because he is fiction and the author who created him will bend the entirety of the representations to grant his creation the solution to the problem at hand. In reality, even granted extraordinary powers of reason, technical skill, and an overwhelming desire to find a solution (things we don't all always have), we have difficulty perceiving things other than what we expect to perceive. We have certain things in common with all humans, yes, but we are also the product of the extensive series of events that led us to this point, events that may not necessarily have prepared us to understand end users who are very different from ourselves.

As Payne suggests, the position that "mental models are essentially analog, homomorphic representations" is a strong one. In other words, we create mental models, maps between cause and effect, that are similar in form to the actual causalities but not necessarily the same thereas. Maps are useful up to a point, but they may leave out a great deal of information that could be useful depending on the type of problem we are trying to solve.

We have, in effect, a mental model of the user that takes into account what we expect from him or her, reliable only to the extent of our existing knowledge, replete with the assumptions necessary for our forward progress towards the design objective, but not necessarily accurate as to every detail. The map is useful, but it is not the same as the territory.

Likewise, the user forms a model of our design, expecting certain results based on certain actions, without necessarily cultivating an understanding of the underlying structure and origin of said results. Even if there is a great Author watching over us, it's unlike that She will bend the fabric of all that is to make others as we expect them to be, as Doyle did for Holmes.

Therefore, whatever the underlying theory, it is better to set aside preconceptions at a certain stage of design to actually observe users in realistic contexts, as Light & Wakeman report they do. Of course, we can take a variety of shortcuts based on the observations and experiences of others who've come before -- 'stand on the shoulders of giants' (provided you trust them).

When there are design choices that cannot be evaluated with test groups to the extent that we might like, or a lack of feedback for some other reason, we can choose to err on the side of respect for the user when that is possible. We will, of course, sometimes show respect in ways that would be appropriate for us individually but aren't necessarily going to work for the audience -- all we can do is try, and do the best we can based on the information we have, within the parameters available (time, space, budgets).

The multiple presentation principle, for example, gives the designer a little more work, but demonstrates respect for the possibility that different people may learn or process differently. We do the work, and minimize the burden on the user. Of course, NOT giving someone work is, in some contexts, not respectful at all, and we get to this in the theory of not doing: "minimizing the mental effort of learners is not necessarily or always a good instructional strategy."

It means that, in design, as in other forms of writing, an understanding of the audience and of our purpose(s) is highly valuable. As we broaden the purposes of our designs, we need a broader understanding of our audiences -- not to categorize them more broadly, but rather to consider their ranges of needs and expectations.

Traditional design overrelies on the normative effects of a standardized stimuli, creating problems for users who perceive or think about that stimuli in ways that diverge from the mean. We have the option of making that same mistake over and over, or of trying something that might make more work for us and, there is a chance, might not work quite as we expect. We weigh the costs and benefits of designing with the success of all end users in mind.