Human Input to Computer Systems: Theories, Techniques and Technology
Mark Billinghurst ,
Abigail Sellen &
© 1994, 2011 William Buxton
In 2007 I published a book, Sketching User Experiences: Getting the Design Right and the Right Design. The objective of that book was to emphasize the importance of bringing traditional design skills to interaction design, or the design of user experience. Part of the motivating factor was that while the skills that were typically being applied to the practice, those from human factors, computer science, and psychology, for example, were essential, but not sufficient. My belief was that the skills that had been developed over years of practice by architects, industrial designers, and graphics designers, for example, could be productively adapted to the practice of user experience design. The resulting book tried to show how, and how such practice could fit into the other interdependent pillars of product design, namely technology and business.
The book, therefore, had to concentrate on this aspect of design practice. Even then, it was arguably too long and too broad. This was frustrating for me, since I knew that what I was leaving out was really important. Furthermore, by placing so much emphasis on sketching, and the application of concepts derived from traditional design, I was worried that I might be swinging the pendulum too far in the other direction. The key point was, the techniques derived from traditional design were meant to augment, not replace, the other disciplines being used. I firmly believed, and still believe, in the notion of the interdependent, interwoven "renaisance team", with the triumverate of design, technology and business at its core.
Given my level of concern that by my emphasis I might be providing a distorted view of my own perspective, on pages 124 and 125 of that book, I made a kind of token hand-wave (seen in the image below), whose intent was to let the reader know that I knew that what I was writing was not the whole story.
My intent was to say something like, "It is just as important to be able to sketch basic theoretical models of your idea on the back of an envelope, as it is to be able to sketch out some literal representation it." The sketching book was about the latter. That was all I had time and space to address there. This current book is about the former. Together, they represent what I see as two key building blocks that make up the skills of a rounded designer, or design team.
In many ways, this whole conversation is to frame what I have been attempting to do in this current manuscript, within the context of what I did in the sketching book. This book is an attempt to capture the essence of the skills that are reflected from the back of the envelope in the image. What I discuss here can certainly benefit from the knowledge - such as it is - gained from the sketching book. But by the same token, my hope is that what I discuss in this book will not only help inform the decisions made in the sketching and early design phase, it will help one start and work at a higher level by applying the theories, models, and techniques to help evaluate concepts and make more informed choices about what, where, when and how to sketch.
To me, combined the two books are kind of the ying and the yang of what I would describe as making up a reasonably rounded design professional - for this class of design, at least. I hope that it accomplishes that, and hope that feedback from this draft manuscript contributes to it doing so. Thanks in advance.
A Work in ProgressThe following is a working draft of a book which I am (far too) slowly trying to complete. The manuscript began as a collection of notes, rough sketches, and reprints which I used in tutorials that I have given, mainly at past SIGCHI and SIGGRAPH Conferences. Some sections of the manuscript remain in point form. Others are near complete. Some of the material has been published in the form of articles, but a lot of other material has not.
While incomplete, and full of rough spots, I am making the manuscript available on the web. I am doing so because it has material in it that I think is of value, not available elsewhere. I undertook the initiative in order to help raise the state of the art. It is not going to do so between now and the time of completion sitting on my hard drive where nobody can get access to it.
I hope that the quality and quantity of information helps to compensate for the unfinished nature of the work. And, since there is no free lunch, the “price of admission”, of early access to the manuscript is my request that the reader will provide constructive comments that will help improve the final version.
1. An Introduction to Human Input to Computers
2. An Illustrated Tour
3. Alternative Perspectives
4. Taxonomies of Input
5. Case Study 1: Touch Tablets
6. Case Study 2: Chord Keyboards
7. Theories, Models & Basic Concepts
8. Human Performance (Shumin Zhai)
- Keystroke Model
- Fitt’s Law
- Steering Law
- C:D Ratio
- Cognition and Human Information Processing
- Chunking and Phrasing
- Memory, Motor Action, and Skill
9. Some Representative 2D Tasks
10. A Collection of Innovative Techniques
11. Two Handed Input (Abigail Sellen, Yves Guiard & Bill Buxton)
- eye gaze input
- force feedback
12. 3D Input
13. Marking Interfaces
14. Gesture Driven Input (Mark Billinghurst)
15. The Future and Emerging Potential
16. References and Bibliography
Input is important, and deserves considerable attention. But once this need is recognized, the intrepid investigator quickly runs into a brick wall. Useful information relevant to the designer is near impossible to obtain. The standard texts on computer graphics (such as Newman & Sproull, 1979, and Foley & Van Dam, 1982) give us general descriptions of devices, both physical and virtual, as well as interaction techniques. Sherr (1988) gives a detailed and comprehensive introduction to input technologies. There has also been a great deal of experimental work in evaluating and comparing input technologies. Milner (1988) and Greenstein and Arnaut (1988), for example, give a fairly comprehensive reviews of the literature.
However, the "real world" is not general. It is made up of real problems, where the designer has to find some match among a mix of users, display technologies, tasks, context and input technologies. Each situation is different, and requires a deep understanding of the relevant factors. But where does one go to obtain this understanding ?
In our opinion, there is no single source, nor even any manageable small collection of sources. While we would love to address this problem once and for all, space, time – and not least of all our own limitations - do not allow us to do so. Thus, we will try to do the next best thing.
In what follows, it is our goal to talk
- albeit briefly - about some of the main issues as we see them.
But perhaps as much time will be spent identifying the issues and how to
approach them, as in giving definitive answers. This way, our intrepid
adventurer has some advance knowledge of the pit-falls and the pots of
gold that lie in wait, as well the basic tools to mine them.
The research behind this work has been mainly funded by the Natural and Engineering Research Council of Canada. Their ongoing support has been more than appreciated. Additional support has also been from Rank Xerox EuroPARC and Xerox PARC, both of which have encouraged the work and enabled me to pursue it on "company time." Likewise, Alias | Wavefront and SGI. have supported me during much of the preparation of this manuscript. Finally, much of the research from our lab at the University of Toronto has been supported by generous grants from Xerox Canada, Apple Computer and Digital Equipment Corp. The efforts of Joy Mountford of Apple and Richard Rubenstein of DEC in making the Apple and DEC support possible is gratefully acknowledged.
Finally, it is important to mention the
SIGCHI and SIGGRAPH conferences, whose conference tutorial programs provided
a forum for me to develop the structure of the material. The opportunity
that that provided, and the comments of the number of tutorial attendees
on the material has made a significant contribution to the form of the