Wednesday, September 23, 2009

Clippy: Misunderstood animated pedagogical agent or spawn of Satan?


Let me help you with that. Oh, come on. I don't want anything. I just want to lend a helping hand. Look at me, I have eyebrows! I need attention. But that is all I need. Feed me attention and I will solve all your problems. It looks like you're writing a letter. I love writing letters. I love reading letters. I just finished reading The Collected Letters of Van Gogh in three volumes. That man could write a letter. Plus, he could paint. But you, look at you. You can't spell. I have to AutoCorrect most of your words. Don't be mad, I have eyebrows! It looks like you're writing a BORING letter. Let me spice it up with quotes from Vincent's letters to Theo. Did you notice that below my eyebrows are actual eyes? These eyes of mine have seen many things but nothing more pathetic than your attempts to write a letter. Do you think John gives a shit about your problems at work? I'm going to say no. Click F10 and I'll replace that uninteresting, grammatically weak, lexically poor sentence with one that will— Please don't, I have more suggestions. I can change shapes! Look, I can—
(Vanishes.)

— Justin Kahn
Short
Imagined Monologues:
Microsoft Office Assistant: The Paper Clip


Hello. My name is Daniel, and I hate Clippy. [“Hello, Daniel.”] The first step to recovery is admitting the problem. Fine. I admit it. I probably spend too much time obsessing over a recently deceased animated pedagogical agent. To clarify that techie term, these interface agents pop up in “instructional environments” and “draw upon human-human social communication” by “embodying observable human characteristics (such as the use of gestures and facial expression)” (“Pedagogical Software Agents”). But my Clippy contempt hardly seems problematic in a computer culture that loves to hate Microsoft’s maligned office assistant. Distain for Clippy stems from some seemingly irrational social consciousness that has collectively perceived, judged, and condemned him to the deepest circle of Hell. I would not be surprised if our generation’s children were born with this hatred scripted in their DNA, afraid not of monsters under their bed but of Clippy in their documents. [“It looks like you’re having a nightmare…”] Clippy fans, if there are any, keep to themselves, seldom exclaiming happiness at his lame jokes or expressing thanks for his intrusive help. Microsoft may have buried its grotesque creation for the moment, but Microsoft’s patent history tells us that that they won’t give up on animated pedagogical agents. Even if Clippy does not return, it’s not hard to image his offspring popping up in our .docx files soon enough.

It may not be a problem to hate Clippy—whose real name is Clippit, by the way—but Sun Tzu reminds us that to truly defeat the enemy we must “understand the enemy.” This essay attempts exactly that—by situating Clippy culturally, historically, and theoretically across modern media, consumer product development, and human-computer interaction research. By understanding Clippy in context, we can begin to answer questions such as, “Are all animated pedagogical agents bad?” or “How do we prevent future iterations of Clippy?” or even “Was Clippy the anti-Christ?”

Pop Culture Paper Clip

Anyone who grew up in the era of Microsoft Word is bound to have an opinion on the annoying paper clip that excitedly bounced and blinked upon opening the program. Even before starting a document, Clippy would give users a helpful “hint”—nothing actually helpful, of course, like which horse to bet on at the track or how to pick up women in a bar. Clippy’s advice usually ranged from program functions to design tips—e.g., keyboard shortcuts for “cut” and paste or how to insert clip art to spruce up a page. Moving past the advice and into the document, I assume most users thought they were through with the strange animated being, but Clippy’s persistence and omnipresence is where this digital office supply went from unusual help interface to infamous rage agent.

Upon typing the salutation to a letter, for instance, Clippy would pop up—like a Viagra advertisement on a sketchy website—and proclaim, “It looks like you’re writing a letter” [“No way, paper clip.”] and ask if you’d like help. Of course, it wasn’t just letter writing that prompted a visit from the Clipster. Any time Word determined that the user needed assistance, based on Bayesian probability, the office assistant would appear (“Office Assistant”). This barrage of help, though not that frequent, was enough to create mass hatred.

A CNET news article from 2001, titled “Microsoft ‘Clippy’ gets pink slip”—six years before Microsft actually canned the clip—quotes Ketan Deshpande, senior software engineer at Manage.com: “Not one person in my office, from the receptionist to the sales people to the engineers to the CEO use the blasted paper clip. Not even my wife, who is an elementary school teacher, uses it” (Luening). It’s not unfair to say Deshpande’s sentiments were representative of the time. Even Microsoft sought to capitalize on the unpopularlity of its office mascot, launching officeclippy.com in 2001—a site with animations of Clippit voiced by Gilbert Godfried where users could commeserate about the horrors of the office assistant and learn how to disable his “helpful” nagging (“Office Assistant”).

The mark of pop culture significance, though, comes with a mention on Family Guy—Fox’s “comedy of references” that targets the obscure, random, or painfully obvious. In the episode “Lois Kills Stewie,” matricidal baby Stewie breaks into the CIA to gain control of the world’s power grid. Suddenly, Clippy appears, saying, “Looks like you're trying to take over the world. Can I help?” To the excited cheers of millions of fans, Stewie replies, “Go away, paper clip! Nobody likes you!” (“Lois Kills Stewie”). Clippy has also made appearances in the comedy of Dmitri Martin, College Humor, Drawn Together, and The Simpsons (“Office Assistant”).

Microsoft’s Frakensteinian Obsession

As humorous or pitiful as he may be, Clippy’s pop culture idolatry was an unintended consequence of his flawed design. Unlike Spud McKenzie or Ronald McDonald, whose sole purpose was advertising, the neurotic paper clip we’ve come to despise was actually meant to help us. To see how, it’s important to understand the history of Clippy’s development, and Microsoft’s obsession with animated pedagogical agents.

In 1987 John Sculley, former CEO of Apple, wrote a book called Odyssey in which he described a system of retrieving information with the help of user interface agents—essentially “people” that would serve as liaisons between the user and the information at hand. Apple shot several promotional videos of their idea that featured a bowtie-wearing butler happily fetching information about various topics, informing the user of incoming phone calls, and generally assisting in data gathering and transferring. The concept as a whole was called the “Knowledge Navigator” (“Knowledge Navigator”).

Bill Gates saw potential for this concept and in the July 1995 edition of InfoWorld magazine extolled the virtues of “social interfaces” or interfaces that allows users to communicate with them using basic social skills like talking and gesturing instead of new commands or difficult logic. Gates predicated that social interfaces were the next iteration of computing and not just experimental toys (Pontin 29).

At that time, Microsoft was banking on the success of its face-lift software for novice users of Windows 3.1. Meant as a competitor to Apple’s At Ease and Packard Bell’s Navigator (“At Ease”; “Packard Bell Navigator 3.5”), Microsoft’s Bob was a GUI designed to look like a house. Users could decorate their house however they pleased. Different icons, representing programs, could be placed in corresponding places throughout the house. Tying all of this customization together were Microsoft’s assistants—mainly a yellow dog named Rover—that let the user ask questions and seek advice about operating and navigating Bob (“Microsoft Bob”). In theory, Bob would let users intimidated by the Windows GUI play around in a more familiar environment through a more social interface.

Not surprisingly, people were not interested in asking a dog questions about their house, and Bob failed almost from the beginning (Newman). But Microsoft wasn’t deterred. They were certain that social interfaces, with animated pedagogical agents like Rover, were the future of interface design. Shortly after Bob’s release, Microsoft also introduced the world to Comic Chat and V-Chat, two GUI chat rooms where users were represented as animated avatars and could change the mood, setting, or method of communication with basic commands (Kurlander, Skelly, and Salesin 2-9; Damer). V-Chat also relied on automated “bots” to host the chat rooms and update users on the room’s status or other news. Users could interact with the bots, but their answers were greatly limited (Damer).

Even with limited success in the chat room world, where people actually wanted to talk to animated agents, Microsoft finally released Clippy on the world in Word 97. I’m sure his designer, Kevin J. Atteberry, who credits himself with creating one of the most annoying animations in history, will be forever proud and/or ashamed of the achievement (Atteberry). Clippy, of course, was not the only iteration of the office assistant. Other, equally annoying animations included Bosgrove the butler, Genie the…genie, Kairu the dolphin, Max the Macintosh computer, Peedy the parrot, Robby the robot, Dot the red ball, F1 the robot, Merlin the magician, Links the cat, Rocky the dog (who looked a lot like Rover), Mother Nature, Einstein, Shakespeare, and even an anthropomorphic Microsoft logo (“Office Assistant”). The default and most iconic assistant was Clippy—whose code name at Microsoft during the design phase was “The F****** Clown” (Sinofsky).


Even during production, programmers realized the pain and suffering embodied in Clippit, but Microsoft executives insisted, with the passion of mad scientists, that these animated agents would be helpful for users —even outside the social interface realm. In 2000, despite the cultural climate of disapproval and growing hatred of Clippy, Microsoft introduced Agent, speech recognition and text-to-speech software that used animated agents as its main user interface. To very few people’s surprise, the software will no longer be supported in Windows 7—a testament to its unpopularity (“Microsoft Agent”). Newer versions of Microsoft’s speech recognition software are not designed around user interface agents.

In the face of user disapproval and financial loss, Microsoft fanatically pursued animated pedagogical agents. Rover even continued his prominence in Windows XP as the search dog. Currently, Microsoft holds at least twelve patents for different animated agents, the latest of which was filed in 2006 (McCracken). But why? What does Microsoft know that everyone else apparently disagrees with? Reviewing relevant HCI research may provide some answers—and the names of a few people to burn in effigy.

HCI Researchers and Clippy, Sitting in a Tree...

Nass, Steuer, and Tauber are widely credited with bringing social interfaces and animated pedagogical agents to the forefront of interface design. In their article, “Computers Are Social Actors,” for CHI ’94, the authors found that users anthropomorphized the computers they were using, applying social rules of politeness and even seeing the computer as having a “self” (Nass, Steuer, and Tauber 77). The authors concluded that even the slightest resemblance of human characteristics was enough for users to interact socially with computers and that a “photo-realistic, full-motion video, or other high-bandwidth representation may be high overrated” (77). Nass et al. expanded on the notion of human-computer social interaction in a CHI ’95 paper titled, “Can Computer Personalities be Human Personalities?” Their study confirmed that humans respond socially to computers and to quote at some length:

In contrast to the prevailing idea that the creation of personality requires natural language programming, artificial intelligence, complex graphical environments, or richly defined agents, this research demonstrates that even the most rudimentary manipulations are sufficient to produce powerful effects. (Nash et al. 229)

From then on, the dominant research in animated pedagogical agency focused on believability of actions instead of believability of representation. Lester and Stone’s “Hermann the Bug” implemented a competition-based sequencing engine with fifteen behaviors vying for dominance. The behaviors are chosen based on the demand of the moment—e.g., if a student waits too long to choose an item in Hermann’s environment, he would reassure them that none of the choices were wrong (Lester and Stone 5-6).

Johnson, Rickel, and Lester called for more study of the role of animated agents in pedagogy, arguing that many features still needed proper testing to determine if they were pedagogically relevant (Johnson, Rickel, and Lester 73). Moreno et al. took this call seriously and wanted to see if using animated agents directly correlated with a better education in computer-mediated environments (Moreno et al. 177). Their study showed that students liked their lessons better with animated pedagogical agents and that students had a greater understanding of material when presented in a social agency environment compared to simple text and images (209). Additionally, one of their studies confirmed Nass, Steuer, and Tauber’s earlier findings that the visual appearance of the agent did not significantly affect student learning (210). Atkinson’s study affirmed the efficacy of animated agents in an electronic pedagogical medium that was originally presented by Moreno et al. (426). Interestingly, Atkinson’s study used a parrot animation very similar to Microsoft’s Agent.

While this short snippet of literature is not by any means exhaustive, I argue that it’s representative of the academic culture at the time—both positive and hopeful about the possibilities of animated pedagogical agents. As far as research goes, there’s not much “out there” to contradict the optimistic outlook on user interface agents in pedagogical realms. And why should there be? The literature makes sense; it is well argued and intricately researched. Why, then, did we have so much trouble with that damned paper clip?

What did Clippy Teach Us?

I suppose I’m forced to admit that the idea of animated pedagogical agency isn’t inherently bad. Clippy’s original sin was not so much in his being but in his designed demeanor. I can theoretically imagine an interface agent that doesn’t compel me to injure it or myself, but its characteristics would need to be drastically different. For instance, Clippy threw the technology in our face. He was always there, like a frustrated school master watching over our shoulder—but with the personality of a used car salesman. For a lot of users, he broke the already fragile illusion of transparent computing. [The computer knows best. Big Clipper is watching you.] The invasiveness wasn’t conducive to learning, especially in a user’s home, an environment where most people want as much control as possible.

Yet it seems reasonable to me that people treat their computers like social actors, so the idea of a social animated agent doesn’t have to be annoying. But consider Clippy as a being. He’s analogous to that annoying guy at work who always knows more about computers than you do—that Jimmy Fallon, Saturday Night Live skit character whose first response to your technical issue is, “Move!” Of course, Clippy tries to help, but so might your well-meaning but overbearing and incompetent cousin who “took a computer class in high school” and once worked on his former girlfriend’s Apple IIe. [“It looks like you’re trying to insert pictures. Would you like help?” he asks. And three hours later you need a new graphics card. And a shower.] Clippy’s persona, and the persona of most of Microsoft’s animated pedagogical agents, simply isn’t up to par with the people I’d want to hang out with on a regular basis. I certainly don’t want them as teachers.

Or it may be that characters like Clippy are the last vestige of the social interface design—a concept that has since evolved into social media and cloud computing. The idea may be back with the revitalization of user experience design but certainly as a shell of its former self and, hopefully, without Clippit.

Regardless, Microsoft’s office assistant was the embodiment of an interesting human-computer interaction concept, and animated pedagogical assistants are likely here to stay. Clippy on the other hand, who became a cultural icon and the personification of technological villainy, will be a twisted reminder of a good idea gone bad. [“Get bent, Clippy.”]

Works Cited

“At Ease.” Wikipedia. 27 March 2009. 19 September 2009. [http://en.wikipedia.org/wiki/At_Ease] Web.

Atkinson, Robert K. “Optimizing Learning From Examples Using Animated Pedagogical Agents.” Journal of Educational Psychology. 94.2. (2002): 416-427. Print.

Atteberry, Kevin J. Odd is Good. 19 September 2009. [http://oddisgood.com/pages/cd-clippy.html] Web.

Damer, Bruce. “Microsoft V-Chat: Frantic Antics.” Avatars! 1997. 19 September 2009. [http://www.digitalspace.com/avatars/book/fullbook/chww/chww7.htm] Web.

Johnson, W. Lewis, Jeff W. Rickel, and James C. Lester. “Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments.” International Journal of Artificial Intelligence in Education. 11 (2000): 47-78. Print.

Kahn, Justin. “Short Imagined Monologues: Microsoft Office Assistant: The Paper Clip.” McSweeney's. 21 April 2005. 19 September 2009 [http://www.mcsweeneys.net/2005/4/21kahn.html]. Web.

“Knowledge Navigator.” Wikipedia. 7 July 2009. 19 September 2009. [http://en.wikipedia.org/wiki/Knowledge_Navigator] Web.

Kurlander, David, Tim Skelly, and David Salesin. “Comic Chat.” Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH. (1996): 225-36. Print.

Lester, James. C, and Brian A. Stone. “Increasing Believability in Animated Pedagogical Agents.” Proceedings of the Eighth World Conference on Artificial Intelligence in Education. (1997): 16-21. Print.

“Lois Kills Stewie.” Family Guy. Fox Broadcasting Network. 11 November 2007. Television.

Luening, Erich. “Microsoft ‘Clippy’ gets pink slip.” CNET News. 11 April 2001. 19 September 2009 [ http://news.cnet.com/2100-1001-255671.html] Web.

McCracken, Harry. “The Secret Origins of Clippy.” Technologizer. 2 January 2009. 19 September 2009. [http://technologizer.com/2009/01/02/microsoft-clippy-patents/] Web.

“Microsoft Agent.” Wikipedia. 22 September 2009. 22 September 2009. [http://en.wikipedia.org/wiki/Microsoft_Agent] Web.

“Microsoft Bob.” Wikipedia. 3 September 2009. 19 September 2009. [http://en.wikipedia.org/wiki/Microsoft_Bob] Web.

Moreno, Roxana et al. “The Case for Social Agency in Computer-Based Teaching: Do Students Learn More Deeply When They Interact With Animated Pedagogical Agents?” Cognition and Instruction. 19.2 (2001): 177-213. Print.

Nass, Clifford, Jonathan Steuer, and Ellen R. Tauber. “Computers are Social Actors.” Human Factors in Computing Systems. (1994): 72-78. Print.

Nass, Clifford, et al. “Can Computer Personalities be Human Personalities?” CHI ’95 Short Papers. (1995): 228-229. Print.

Newman, Michael. “Bob is dead; long live Bob.” Post-Gazette. 23 May 1999. 19 September 2009. [http://www.post-gazette.com/businessnews/19990523bob6.asp] Web.

“Office Assistant.” Wikipedia. 17 September 2009. 19 September 2009. [http://en.wikipedia.org/wiki/Office_Assistant#cite_note-13] Web.

“Packard Bell Navigator 3.5.” Nathan’s Toasty Technology. 19 September 2009. [http://toastytech.com/guis/pbnav35.html] Web.

“Pedagogical Software Agents.” Communications of the ACM. 47.4 (2004): 47. Print.

Pontin, Jason. “Microsoft plans social interfaces everywhere.” InfoWorld. 23 July 1995. 29. Print.

Sinofsky, Steven. “PM at Microsoft.” Steven Sinofsky’s Microsoft Tech Talk. 16 December 2005. 19 September 2009. [ http://blogs.msdn.com/techtalk/archive/2005/12/16/504872.aspx] Web.

No comments: