HCI due for a quantum leap?

Christian Crumlish  

September 05, 2006

Related Topics

At ACM Queue, John Canny, the Paul and Stacy Jacobs Distinguished Professor of Engineering at UC Berkeley, writes about the future of human-computer interaction. For many years HCI has been evolutionary, not revolutionary. Is this about to change?. He begins by making a case for the centrality of HCI in product design:

[I[t’s not a good idea to separate “the interface” from the rest of the product, since the customer sees the product as one system. Designing “from the interface in” is the state of the art today. So HCI has expanded to encompass “user-centered design,” which includes everything from needs analysis, concept development, prototyping, and design evolution to support and field evaluation after the product ships. That’s not to say that HCI swallows up all of software engineering. But the methods of user-centered design – contextual inquiry, ethnography, qualitative and quantitative evaluation of user behavior – are quite different from those for the rest of computer engineering. So it’s important to have someone with those skills involved in all phases of a product’s development.

He goes on to suggest that the next leap forward in HCI will involve “context”:

Let’s start with the cellphone. It has a tiny screen with tiny awkward buttons and no mouse. From start to finish, it was designed for speech. The microphone and speaker are small but highly evolved, and the mic placement in its normal position is optimal for speech recognition. We’ll get to speech interfaces shortly. If it’s a smart phone, it probably also has a camera and a Bluetooth radio. It has some kind of position information, ranging from coarse cell tower to highly accurate assisted satellite GPS.

This is all “context” information, in contrast to the “text” you might type on the keyboard or see on the screen. Normally, WIMP interfaces rely entirely on the text you type (let’s include mouse input) to figure out what to do. Context-aware interfaces use everything they can. This is particularly relevant to mobile phones. When you’re using a phone, you’re either in some “place” (cafe, restaurant, store) where you do rather specific activities, or you’re moving between places. If the phone can figure out what that place is, it can also provide services that you want there, or that complement services that that place provides (e.g., song previews in a music store, comparison pricing in a supermarket, stats or replays at a baseball game). When you’re between places, the phone can use other pieces of context to figure out what services to offer, or it can wait for you to ask.

Let’s work through a concrete example: It’s 7 p.m., it’s raining, and you’re walking in San Francisco (you’re from out of town). You open your phone and it displays three buttons labeled “Dinner?”, “Taxi?”, and “Rapid transit?”. Selecting “Dinner?” will present restaurants you’re apt to like (using collaborative filtering) and even dishes that you may want. The other options leverage the fact that the phone “knows” that you aren’t driving and that it’s raining. It also selects “Rapid transit?” (using that name rather than BART as locals know it, since you’re not local), rather than bus or tram options since it knows your destination and/or because BART is easier to figure out for out-of-towners than the MUNI bus and tram system. The system’s “smarts” are built on knowledge of other users’ behavior, knowledge of your own behavior history and preferences, and the immediate context, which includes time, place, weather, Bluetooth neighborhood, etc. These three pieces represent the three fundamental facets of context that we use in all our work: immediate context; activity context, which is about the history of the particular user and a few others (because many activities are cooperative); and situational context, which is about how other actors typically behave in that situation.

Context-awareness is a dream for marketers. Imagine this: Instead of the user initiating the request for “Dinner?”, the phone beeps and presents a message, “Aqua restaurant (a leading San Francisco seafood restaurant) is two blocks away and has a special on salmon-in-parchment for $20.” Now, I’m a very rational person, but I also have a weakness for the pink fish, and when I’m tired and wet and I see that, it really doesn’t matter what the other options are. That is an example of a proactive service, which if executed right, should be a boon to both consumers and advertisers. Before you raise the specter of a Minority Report-style advertising assault, I should tell you that I don’t expect to let just anyone send that kind of message to my phone. I’m going to charge a lot for that (probably in whole dollars), so an advertiser had better be very sure of a conversion before trying it. If so, then I am likely to use that service at that time, and then it’s very useful to me. If Aqua restaurant beacons this message to a few seafood-loving out-of-towners in the neighborhood that night and gets two or three conversions, then the restaurant will be ahead. If I get a half-dozen of those in an evening and one of them gives me a good service, then I feel like I’ve won. If none of them works out, well then at least I’ve earned my BART (rapid transit) fare home, and some change.

It’s a long article but well worth reading the whole thing.