Human Computer Interaction Consortium


HCIC 2001 > Papers

Why HCI Fails

Invited Speaker: Don Norman, UNext Learning Systems and Nielsen Norman group


"Why," not "If "or "Where" or "When."

I contend that as a practical discipline, one intended to make a difference in the real world of products, HCI fails, and it fails grandly.

One has only to look at any product, or the Florida voting problem, or to talk to an HCI professional off-line, in the bar, when they aren't touting the company line. HCI gets no respect. It isn't listened to, not even in the very best of situations.

Sure, you will tell me of the tens of millions of dollars AT&T saved through careful reshaping tasks. Sure, you will show little improvements here and there. But these are tiny, insignificant, and probably not even HCI successes anyway.

HCI fails for several reasons.

Practitioners are badly trained. If trained in the University, they learn to critique, to find problems. They also learn how to study users and determine their true needs. They haven't a clue about how to design. And design is where the action is. Well, maybe engineering, marketing. and sales. But not HCI.

The HCI community thinks usability is the goal. It isn't, it never will be, and it shouldn't be. Usability is always secondary, sometimes tertiary - and properly so.

HCI is not a stakeholder in the product process. HCI is a service, often a money sink, a cost center. HCI does not bring in revenue to the company, so it is ignored. Oh, sure, HCI can save money in service calls, in repeat sales, in time on task and all that stuff, but that is not what CEOs and GMs think about when they think about their product line, their P&Ls, and their own promotions.

You want examples? Lots. Here are some trivial ones: The All-CAPS problem on the keyboard. Trivial, but telling. The power switch on the Mac is one I have already written about. You want a big one? The true story at Apple. Or even Microsoft and IBM, the two best companies I know of in the usability field today. The inside story at almost every company is yet another. The Florida voting problem is another -- the problems with those machines, with butterfly ballots, and with voting in general have been well researched and known by professionals for a decade. Nobody cared enough to change things (and why should they?).

In her talk, Judy Olson will show how HCI has succeeded in company after company. Without knowing anything about her talk, let me categorically say, her findings do not dispute mine, even if she thinks they do. I bet they wee small successes, and often as not, accompanied by much frustration and internal politics (which they not have talked to Judy about). To make a difference, we need to change. Educate differently. Have a different attitude about usability, and get into positions of real power. How do we do this? Jakob Nielsen seems to be hated by most HCI professionals, but he is doing more to help them then anything ever done. His sound bites attract attention. They work. Frames suck. WAP is useless. Flash -- 99% bad. Sound bites that catch the imagination. WAP actually got people talking about usability in public. The Florida vote might do the same. Jakob argues that the web is the best ground for usability. He may be right, yet look at how few studies academics (or HCIC) dos on web-related issues. ACtually, look at how few studies HCI or HCIC does on design.

So wake up, folks. We aren't much respected in either Psychology or Computer Science. And we aren't much respected in industry. So why are we doing this?

Achieving Speed in Globally Distributed Project Work

David Atkins, Lucent Technologies
David Boyer, Avaya Research Labs
Mark Handel, University of Michigan
James Herbsleb, Lucent Technologies
Audris Mockus, Lucent Technologies
Graham Wills, Lucent Technologies

Abstract: We used a survey and an analysis of change management data to measure the extent of delay in a multi-site software development organization. Results indicated that cross-site work takes much longer and requires more people than comparable same-site work. We describe several tools we have deployed and others we are developing as new products, all of which are designed specifically to address the issues of speed we identified. We conclude with an examination of research findings that were helpful, others that were less helpful than we would have thought, and research issues we think should get more attention.

Download slides: PowerPoint | HTML

Discussant: Pamela Hinds, Stanford University

Evaluating software architectures for usability: Methods and application

Len Bass, Carnegie Mellon University
Bonnie E. John, Carnegie Mellon University

Abstract: In this paper, we present an approach to improving the usability of software systems based on software architectural decisions. We make specific connections between aspects of usability such as the ability for a user to "undo" or "cancel" and software architecture. Designers can use this collection both to generate solutions to those aspects of usability that they choose to include in their system and to evaluate their systems for specific aspects of usability. We present the results of using this approach in evaluation and design of three real-world systems.

Download slides: PowerPoint | HTML

Discussant: James Herbsleb, Lucent Technologies

Download discussant slides: PowerPoint | HTML

HUTCHWORLD: Lessons Learned

Shelly D. Farnham, Microsoft Research
Linda Stone, Microsoft Research
Lili Cheng, Microsoft Research
Melora Zaner-Godsey, Microsoft Research
Karen Syrjala, Fred Hutchinson Cancer Research Center, University of Washington
Janet Abrams, Fred Hutchinson Cancer Research Center, University of Washington
Ann Marie Clark, Fred Hutchinson Cancer Research Center

Abstract: The Virtual World Group (VWG) in Microsoft Research and the Fred Hutchinson Cancer Research Center (the Hutch) are collaborating to develop and study online social support systems for patients at the Hutch. Medical research has shown that social support contributes positively toward psychological and physical well-being. However, in the real world, it is difficult for immune-compromised patients, their families, and their caregivers to interact with others facing similar challenges. In order to address the needs of Hutch patients and their caregivers, Microsoft and the Hutch developed HutchWorld, a Virtual Worlds application, to provide online, computer-mediated social and informational support. The present paper describes the approach of each design phase of the project, and lessons learned about a) developing an interactive online support system for a specific audience, b) the deployment of the online social support system to its target audience, c) the rigorous study of both its use and its impact on user well-being, and d) issues affecting collaboration among technology, medical, and research groups.

A controlled study examined the impact of HutchWorld. Self-reported evaluations of HutchWorld and analyses of usage patterns from user logs will be discussed.

Discussant: Stuart Card, Xerox PARC

Design for Extreme Collaboration

Gloria Mark, University of California, Irvine
Paul DeFlorio, Jet Propulsion Laboratory

Abstract: In this paper, we describe the real work experiences associated with a radical new form of working together, extreme collaboration, where team members are physically collocated, and technology is intensively used to support their coordination and information exchange in synchronous work. We describe the role of technology in different phases of the team process, and an experiment using life-size HDTV to support distance collaboration with this team.

Discussant: Terry Winograd, Stanford University

Download discussant slides: PowerPoint | HTML

Design as a Minority Discipline in a Software Company: Toward Requirements for a Community of Practice

Michael J. Muller, Lotus Development Corporation
Kenneth Carey, Lotus Development Corporation

Abstract: We describe our participatory analysis of the diversity of working relations and roles of designers at Lotus Development Corporation. Designers are an example of a minority discipline - that is, a discipline whose members are often isolated in their work teams among coworkers with different training, backgrounds, and career paths. We contrast designers' experiences at Lotus with conventional results from design research of the work of groups of designers.

Discussant: Terry Roberts, Sun Microsystems

What's going on out there? What major companies are doing about User Experience Engineering.

Judith S. Olson, University of Michigan

Abstract: To see what the major companies are doing about usability, I interviewed 10 people from 10 companies -- a mix of large software developers, telecommunications companies and web and consulting organizations. They talked about what they did, how they gain acceptance, their key methods, and various success/disaster cases supporting the use of User Experience Engineers in the product development lifecycle. This paper reports on the findings and sets an agenda for research and practice.

Download full paper: PDF | PostScript

Discussant: Bonnie E. John, Carnegie Mellon University

Welcome to the Wireless World: Problems Using and Understanding Mobile Telephony

Leysia Palen, University of Colorado, Boulder
Marilyn Salzman, QWEST Advanced Technologies

Abstract: Wireless devices, like PDAs and mobile phones, call for comprehensive treatment of their usability. As mobile technologies, wireless devices can be used in a variety of places, and therefore must be dynamically managed: software and hardware features have constantly changing relevance depending on the situations in which they are employed, for example. Like other service-based technologies, understanding how wireless devices operate extends beyond the hardware and software components to include an understanding of network services ("netware") and service provider business agreements ("bizware") as well. Finally, wireless devices, and in particular wireless phones, play a role in the social world. Developing a personal practice of wireless telephony communications necessarily includes attention to how one affects and is affected by one's social networks and locale. Usability of such technologies, therefore, is more accurately assessed if it extends beyond an examination of the devices themselves to include broader attention to the contexts of their acquisition and use.

We report on the results of a study in which 19 new wireless phone users were closely tracked for the first six weeks after service acquisition. The proposal for the study originally called for an assessment of the usability of telephone handsets, and for an evaluation of the process of learning and discovery of hardware and software features. However, we kept the investigation broad in scope, hypothesizing that there was a great deal to learn about the experiences of new users more generally. As we observed our new users, we discovered that fundamental aspects of sales, service, and business communications were critical to their learning and discovery of features. When these aspects were misunderstood by users, it directly affected the use and usability of their wireless telephones.

Discussant: Daniel Gruen, Lotus

Suspending Disbelief - Selling Low Fidelity Studies to Product Developer

Nancy Shepard, Sun Microsystems, Inc.

Abstract: Low-fidelity prototypes and simulations are quick low-cost methods for studying user behavior and gathering design feedback early in the product development lifecycle. Yet development teams are often resistant to using these low- tech methods for several reasons:

  • automatic conditioning to use higher quality prototypes
  • limited experience with low-fidelity models
  • judgement of colleagues and management
  • disbelief that quality information can be gained from low fidelity methods

Discussant: Jonathan Grudin, Microsoft Corporation

The Real Challenges for MSN Usability at Microsoft

Gayna Williams, Microsoft Corporation

Abstract: The purpose of this paper is to highlight the very real challenges faced by usability engineers at Microsoft, particularly the MSN usability group, in having an impact on the development of commercial software and web sites. The presentation will cover several methods the usability engineers have used and adapted to gather information to impact the products, and why adoption is necessary, while the extended abstract aims to provide a richer context in which to understand challenges that need to be overcome in order to see good research converted into applied results (i.e., the UI).

Discussant: Judith S. Olson, School of Information, University of Michigan

Download discussant slides: PowerPoint | HTML

A Graphical User Interface Toolkit Odyssey

Harry Vertelney, Sun Microsystems, Inc.

Abstract: This presentation reviews an invitational design competition to create the default look and feel for the Java platform. The discussion will revolve around the issues encountered while trying to create a cross platform look and feel in a quasi open source milieu. The discussion will focus on the design brief, competitor submissions, public scrutiny and feedback followed by the difficult decisions we had to make based on polarized feedback from different user communities.

Download full paper: PDF | PostScript

Discussant: Jodi Forlizzi and Scott Hudson, Carnegie Mellon University

Workplace Instant Messaging Prototypes: A Case Study of HCI in the Real World on Internet Time

Steve Whittaker, AT&T Research
John C. Tang, Sun Microsystems Laboratories

Abstract: AT&T Research has conducted studies of Instant Messaging (IM) use in the workplace. Sun Labs and AT&T have created research prototypes that provide IM and other awareness features for users in distributed workgroups. Taken together, our research presents a case study of how studying real use in the world led to design implications for research prototypes. But an even larger question emerges on how to transfer the experience gained in the research community on awareness to the fast-paced world of IM products. AOL and the like appear to be shaping the landscape of mass-market IM products (arguably the most widely deployed interactive CSCW technology). How can and should the research community inject our experience gained from awareness prototypes into the fast-moving market of IM products?

Download full paper: Text
Download slides: PowerPoint | HTML

Discussant: Gary Olson, School of Information, University of Michigan