Figure 1: A model for the performance differences between one-handed input and two-handed input.
Owen, R., Kurtenbach, G., Fitzmaurice, G., Baudel, T. & Buxton, W. (1998). Bimanual Manipulation in a Curve Editing Task. Unpublished manuscript. Toronto: Alias|Wavefront Inc.
Russell Owen, Gordon Kurtenbach, George Fitzmaurice, Thomas Baudel, Bill Buxton
Alias | Wavefront
210 King Street East
Toronto, Ontario
Canada, M5A 1J7
<rowen, gordo, gf, tbaudel, buxton>@aw.sgi.com
416 362-9181ABSTRACT
In this paper we investigate the relationship between bimanual (two handed) manipulation and the cognitive theories of task integration, divided attention and epistemic action. We explore these relationships by means of an empirical study based on a curve editing task. This task was designed on the principle of integrating the visual, conceptual and input device space domain of each hand's individual task. We provide evidence that this task has superior performance over a unimanual (one-handed) version. Furthermore, the experiment demonstrates that as the task becomes harder (i.e., more cognitively demanding) the bimanual technique exhibits even greater performance benefits over the unimanual technique. We argue that the design principles and performance improvements are generalizable to other task domains. KEYWORDS: two-handed input, bimanual input, curve editing, task integration, divided attention
There have been various studies concerning interaction techniques which utilize two handed input. [3, 4, 8, 9]. Three potential advantages of two handed input motivate these studies:
1. Time-motion Two-handed input will allow users to perform tasks faster. Reduced time-motion can be attributed to several things. First, some operations can be accomplished in parallel (such as simultaneously moving and scaling a rectangle). Another time-motion saving may result from a reduction in the amount of task switching. For example, two endpoints of a line can be moved without having to move the cursor back and forth between them.
2. Existing skills Another advantage of two handed input is that there are everyday two handed actions which we can use as a metaphors for computer interactions. The intention is to use metaphors to assist in learning an interaction style. For example, the Toolglass/Magic Lenses interaction technique is metaphorical to holding a painter's pallet in the one hand while drawing or picking with the other hand. This is consistent, in general, with the everyday bimanual skills used in the physical world, as outlined by Guiard [6].
3. Expressiveness This advantage is much more intangible than the previous two and is the subject of this paper. Informally, we define expressiveness as the ability of the interaction technique to allow the user to explore and browse the data being operated on. Thus expressiveness is linked to the ability to quickly manipulate, perceive, and evaluate the transformed data. We give evidence in this paper, that when tasks require more interactive manipulation of the data, two-handed input can result in better performance that one-handed input.
One might say that two-handed input simply increases the degrees-of-freedom of input and therefore will by definition result in better performance than one-handed input. However, we have found that in practice this is not generally true. A classic criticism of two handed input has been the "tapping the head and rubbing stomach" argument. That is, two handed input is too much of a burden on the user. Kabbash et. al. [8] have shown that two hands are not always better than one. In the case where a user is working on a complicated task, two-handed input further complicates the task by forcing the user to coordinate the actions of their hands.
In our research on two-handed input we have been experimenting with many different prototype designs. We have concluded that in order for two-handed input to be effective it must be designed carefully. One obvious and very general two handed design is to have two input devices (for example, two mice, one for each hand) and two cursors. Both time-motion and existing skills advantages seem plausible with this approach. Tasks may take less time because one cursor does not have to "run around and do everything" and the notion of two cursors is metaphorical to our two hands.
However, we have found that in practice this design performs very poorly. In many cases, the user's job boils down to serially switching their focus between two cursors and serially operating each input device. The gain (not having to move one device around to do everything) is outweighed by the cost of keeping track of and switching between two different activities.
One reason for the failure of this design is the divided attention problem (see [10] for additional information on divided attention). When there are multiple sources of information (for example, two cursors) we must make choices about what to attend to and when. This results in additional "switching costs" which outweigh the benefits [11].
We have also noticed that other two handed designs work very well and almost transparently, for example, scaling and moving a rectangle by controlling its two opposite corners.
This case, we believe, is the opposite of the divided attention problem. The operations of moving and scaling are visually integrated and therefore switching costs are minimized.
Integration can also be conceptual. We define conceptual integration as when the user perceives and prefers to think of the operations of the two hands not as separate activities but as a single activity. For example, holding a nail and hitting it with a hammer can be conceptually integrating into the single activity of "hammering a nail". This is similar to Guiard's perspective on bimanual tasks [6]. Jacob provided evidence of conceptual integration effecting computer interaction [7].
Integration can also happen in other ways besides visually and conceptually. The way in which we manipulate the input devices can be integrated or divided. For example, in the two cursors, two mice approach, the input domain is split into two different coordinate systems: one coordinate system for the left (the left side of your desk) and one for the right (the right side of your desk). However, if two absolute mice were used with the same physical coordinate system, we would say the input space was integrated.
Thus, two-handed input is most likely to be effective when task integration is supported and divided attention is minimized.
We also believe that the difference between one-handed and two-handed techniques will not be completely explained by differences in time-motion. See Figure 1. We believe the difference between the two techniques consists of one-handed time motion costs and two-handed cognitive benefits.
We hypothesize that the cognitive benefits of the two handed techniques can be attributed to epistemic action.
There is growing evidence that motor activity can be classified as either
epistemic or pragmatic action [12]. Epistemic actions
are performed to uncover information that is hidden or hard to compute mentally.
The physical actions make internal cognitive computation easier, faster
and more reliable. For example, we sometimes use our fingers when we count.
Or, novice chess players may physically move a chess piece, temporarily,
to its new position to assess the move and possible counter-moves by an
opponent. The epistemic actions can improve cognition by reducing the (1)
memory, (2) number of steps and (3) probability of errors involved during
mental computation. In contrast, pragmatic actions are physical actions
whose primary function is to bring the user physically closer to the goal.
We believe two-handed interaction techniques offer more opportunity for epistemic rather than pragmatic actions compared to one handed techniques. This is because two- handed input inherently provides more degrees of freedom and therefore it is possible to manipulate data more rapidly to uncover information that is hidden or hard to compute mentally. This ultimately results in faster task completion times.
We choose curve editing by manipulating control vertices (CVs) as the
task to test these beliefs about two-handed input. This task is commonly
found in computer graphics applications and can be challenging (See [5] Foley and van Dam for more detail on curve representations).
The difficulty of the task arises from the relationship between the CVs
and their effect on the curve being controlled. Figure 2 show examples of
how the locations of the CV interact to produce uninituitive resulting curves.
While the effects of a CV on a curve is learnable, most users report that
it is very hard to get a particular shape of curve without some trial and
error in placing the CVs.
The task in our experiment required a subject to manipulate two CVs until the curve being manipulated (the response curve) matched a target curve. The response curve was displayed superimposed over the target curve.
We selected this task for several reasons:
1) We are interested in using two-handed input to enhance curve editing. While the task does not truly reflect a practical way to perform curve editing (for example, most curve editing involves the manipulation of more than 2 CVs), we hope that what we learn about two-handed input from this simple curve editing task can be applied to more realistic curve editing situations and different problem domains.
2) We believe that for this simple curve editing task we can create a two-handed interaction technique that supports conceptual, visual and input space integration, thus avoiding some of the pitfalls of two-handed input.
3) We believe that curves which are difficult to match will require high amounts of manipulation. We hypothesize that this condition will bring out the advantages of two-handed input.
The experiments were run on an SGI Indy using a 20 inch color monitor. Subjects sat at a fixed distance of 100cm from the screen. An 18x25 inch multi-device Wacom digitizing tablet was used as the input device. For the one handed case a stylus was used in the right hand. For the two handed case the stylus was held in the right hand and a small puck (approximately 1" in diameter) was held in the left hand. The wacom tablet is capable of sensing both the stylus and puck simultaneously. See Figure 3. The experiment program was written using the X input extensions and OpenGL graphics library.
The task involved presenting a target curve to the user then asking them to match (overlay) the target curve with the response curve by manipulating the control vertices of the response curve. The endpoints of the response curve start at the endpoints of the target curve and require no manipulation. To complete the task the subject manipulated the two CVs which control the shape of the response curve until the curves match within a fixed error for 1/3 of a second. To eliminate the chance that the user would start a trial in random position that was close to a match position, the beginning of each trial was demarcated by the user by placing their hand(s) in a home position and pressing down with the stylus to signal the start of the trial.
The two CVs were distinguished by color, one was red the other blue. When two hands were used to manipulate the CVs, the red CV was assigned to the right hand and the blue CV to the left.
From this basic configuration we tested three input configuration treatments:
1) One hand In this case CVs could be manipulated one at a time using the stylus in the right hand. A cursor tracks the stylus position on the tablet when the stylus was very close or lightly pressed against the tablet. Applying more pressure to the stylus when the cursor was over a CV allowed subjects to "drag" a CV. The entire surface of the tablet was mapped to the screen of the workstation.
2) Two hands, integrated device space In this case no CV selection was necessary since the cursors for the two input devices were automatically attached to the CVs at the beginning of the trial. The entire surface of the tablet was mapped to the screen of the workstation. Both the left and right hands operated in the same coordinate system so when the left and right hands where in the same place physically, the CVs where in the same place on the screen.
3) Two hand, non-integrated device space The two handed non-integrated
case was identical to the previous two handed case except that the tablet
was logically segmented into two halves. The position of the left hand relative
to the left half of the tablet was mapped to the workstation screen. Similarly,
the position of the right hand relative to the right half of the tablet
was mapped to the workstation screen. See Figure 4.
These three treatments were crossed with two output configurations (See Figure 7):
1) target CVs In this treatment the CVs for the target curve were displayed.
These CVs, like the CVs for the response curve where also color coded in
red and blue, but in a deep hue. Thus matching curves in this condition
simply involved moving the red CV over the dark red CV and the blue CV over
the dark blue CV. See Figure 5.
2) no target CVs In this treatment, the target CVs were not displayed. Matching in this condition became a matter of manipulating the response CV until the response curve matched the target curve. See Figure 6.
| CV targets | no CV targets | |
|---|---|---|
A within subject, repeated measures design was used. Thirty trials were run for each condition. Each 30 trials used the same ordered set of target curves. This resulted in 180 data points per subject. Six subjects were run resulting 1080 data points for each condition. Conditions for the each subject were ordered by a 6x6 Latin Square to ensure between condition ordering effects were counterbalanced.
The dependent variable of task completion time was measured for each
trial. Subjects' subjective impressions were collected in an exit interview
and survey. All input device events data (pen up, pen down, pen moved) were
logged to allow separation of selection and movement time in the one handed
condition.
A total of 6 employees from Alias/Wavefront were run as subjects. They
were pre-screened for color-blindness. All subject were right handed 1 .
The calculation of error between curves that matched the user's perception of curve closeness was a non-trivial task. We used the curve error function developed by Thomas Baudel[13]. (see Appendix for a description of the curve matching error algorithm).
H1: The two handed conditions will yield faster task completion times than the one handed conditions.
H2: Even when switching costs are removed from the task completion times for the one-handed conditions, H1 will still hold.
In the one handed case subjects are forced to move CVs one at a time,
switching back and forth between the two CVs (switching costs). In terms
of time motion mechanics switching costs appears to be the major difference
between the one-handed and two handed techniques. However, we believe that
there are additional cognitive benefits in the two-handed condition which
will further reduce task completion times. Therefore, even when switching
costs are factored out, the two-handed technique will still outperform the
one-handed technique.
These cognitive benefits will be described in more depth in the discussion
section.
H3: Assuming that task completion times degrade in the no target CV condition, the amount of degradation will be greater for the one handed conditions than for the two handed conditions.
In other words, we expect two-handed techniques to exhibit more of an advantage as the task become more difficult.
H4: Integrated device space conditions will yield faster task completion times than non-integrated device space conditions.
The integration of the device spaces allow subjects to utilize their sense of relative limb position in the placement of the CVs. This should result in reduced task performance time.
Figure 8 shows the results of the six conditions.
As we predicted, the absence of CV targets had a major effect on task performance F(1,1073) = 19.475, p < 0.01. Thus we conclude that curve matching without the targets was a significantly more difficult task than curve matching with the target CVs.
H1 (two-hands faster than one) was found to be true. Task completion times
were significantly shorter in all the two handed conditions (F(2, 1072)
= 13.685, p < 0.05). On average the two handed condition was approximately
40% faster than the one handed.
H2 (two hands still faster, even after subtracting switching times in one-hand condition) was found not to be true. For each trial in the one handed case, the task completion time was decomposed into stylus up time (the CV switching time) and stylus down time (CV manipulation time). We then subtracted switching time from the total time to complete the trial. However, analysis of this derived data showed no difference between the one handed and the two handed techniques.
H3 (the advantage of two hands becomes more pronounced when the task becomes difficult) was found to be true. An interaction was found between input configuration (one hand, two-hands integrated, two-hands non-integrated) and output configuration (target CVs, no target CVs) F(2, 1069) = 4.534, p < .05.
Figure 8 shows that the source of the interaction is the increase in the magnitude of the advantage of the two handed technique when the task becomes more difficult (i.e, no target CVs). Most notably, the magnitude of difference between one-hand and two hands in the easy condition (target CVs) versus the difference in the hard condition (no target CVs) is approximately factor of three.
H4: (integrated faster than non-integrated). We found no significant differences in task completion time between an integrated and non-integrated device space.
The most obvious result was the decreased task completion time for the two handed case as compared to the one handed case. This result has been observed in other studies [4, 8]. As Kabbash et. al. points out not all two handed tasks give superior performance. If the feedback requires the splitting of visual attention between multiple targets the two handed technique can actually be worse than the one handed. In our experiment the two control points had the potential for splitting attention, however we believe the curve provided a visual integration of the task. We believe this avoided the divided attention pitfall and allowed the two handed technique to demonstrate its advantage.
Although this experiment did not explore the issues of conceptual integration, we believe the task exhibited conceptual integration. Earlier, we defined conceptual integration as when the user perceives and prefers to think of the operations of the two hands not as separate activities but as a single activity. For example, subject may have thought of the task not as the process of manipulating each CV but as the process of reshaping a curve with two hands. In our post-experiment survey we asked subjects if they ever matched the curve by watching only the curve, not the target CVs, even when the target CVs were displayed. While we had experienced this phenomena ourselves, none of our subjects reported a similar phenomena. Perhaps this can be attributed to subjects not being extremely experienced with the curve matching task.
We did not find any evidence of epistemic action since H2 (two hands still faster, even after subtracting switching times from one-hand condition) was not supported. This could be due to an inaccurate measure of switching time. Essentially we defined switching time as the time the subject spent not dragging a CV (the time spent with the pen up). However, a subject might not only be using this time to switch to the other CV but also to think about the problem, rest or visually acquire the other CV. Thus this time may be an over-estimate of pure time motion switching costs and result in the cognitive effects being masked.
One should also note that switching time is an unavoidable cost in the one handed technique (just as deciding which hand to move is an unavoidable cost in the two handed technique). An attempt could be made to optimize switching time (for example, a key that the subject presses to switch between CVs). However, despite how small the cost, switching time will always be present in the one handed technique.
Our results support the notion that two handed input can be effective for difficult tasks. As the task became more difficult in the experiment the relative advantage of the two handed case increased. Leganchuck et. al. [13] have also found evidence of this. Furthermore, the majority of our subject reported preferring the two handed technique over the one-handed technique.
We found no performance difference between integrated and non-integrated input space. This may be due to several reasons: First, because the hand positions were always represented on the screen, hand position was never ambiguous. Thus there was no advantage in sensing the relative hand position in the shared case. Second, the results may also be confounded by collisions or avoidance of collisions between the hands during the task. The benefit of sensing relative hand position may have been canceled out by the cost of hand collisions in the shared case.
Although our experiment showed no significant differences between integrated and non-integrated device spaces, there are practical reasons to choose either approach. If non-integrated device space is used either the device resolution can be split in two, or separate devices used. The first reduces the resolution of the input device and physical size of the input space. The second, consumes twice as much physical ("footprint") space. These problems are not present with an integrated device space scheme. However, device collisions are possible, although interaction techniques can be designed to minimize collisions.
Our two-handed task was not very realistic in the sense that it did not require any selection action by the subject. We have found in experimental prototypes that selection with two devices can be problematic. Two basic schemes can be used: automatic and manual. In the automatic case, the user picks something with one hand and the system automatically assigns another object to the other hand. Alternately, a manual scheme requires the user to explicitly select each object.
Both these approach have pros and cons. Automatic assignment saves the
user time in selection (only one item needs to be selected), however the
major problem is that the system must determine a proper secondary selection
and this is often difficult to predict. The major problem with manual selection
is that the user is required to pick with their non-dominant hand and this
can be difficult. Furthermore, both of these approaches are susceptible
to divided attention. Supporting selection in two-handed tasks is a topic
for future research.
We suggested three design principles for two handed input. In order to support effective two-handed input:
- The task should be visually integrated. The task must not promote divided visual attention between the activities of the two hands.
- The task should be conceptually integrated. That is, the user must be able to conceptualize the operation performed by the two hands as a single operation.
- The task should employ integrated device spaces.
Our experimental task was designed around these principles and demonstrated significant advantage over a one-handed technique. Thus we have shown that two-handed interactions can be designed which do not suffer from the "tapping the head and rubbing stomach" syndrome.
Our experiment was not able to isolate the effect of cognitive benefits in the two-handed technique. Future research is needed to conduct further experiment to tease out this effect, if it exists, and more fully understand it. We are continuing to prototype two-handed interaction techniques. We intend to explore and expand two-handed input design principles.
We greatly acknowledge Wacom Technology Corp. for assistance in utilizing the multi-device mode on their tablets. We also would like to thank Alias employees who served as subjects in our experiment and supplied valuable comments. Beverly Harrison and Shumin Zhai provided us with help in the experimental design and analysis. Steve Spencely provided comments on requirements on curve editing tasks.
1. Bartels, R., H., Beatty, J., C., Booth, K., S., Bosch, E., G., Jolicoeur, P., (1993) Experimental Comparison of Splines Using the Shape-Matching Paradigm, Transactions on Graphics, 12(3), 179-208.
2. Baudel, Thomas, A Mark-Based Interaction Paradigm for Free-Hand Drawing, Proc. ACM Symposium on User Interface Software and Technology (UIST'94), 185-192
3. Bier, E., A.,Stone, M., C., Fishkin, K., Buxton, W., Baudel, T., A Taxonomy of See-Through Tools. Proceedings of the ACM CHI'94 Conference on Human Factors in Computing Systems, 358-364.
4. Buxton, W., & Myers, B. A., (1986) A study in two-handed input. Proceedings of the ACM CHI'86 Conference on Human Factors in Computing Systems, 321-326.
5. Foley, J., van Dam, A., Computer Graphics Principles and Practice Second Edition, Addison-Wesley, New York, 1990.
6. Guiard, Y. (1987). Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. In Journal of Motor Behavior, 19(4), 486-517.
7. Jacob, R. J. K., Sibert, L., McFarlane, D. et Mullen, P. (1994) Integrality and Separability of Input Devices, ACM Transactions on Computer-Human Interaction. 1(1), 3-26.
8. Kabbash, P., Buxton, W., and Sellen, A., (1994) Two-Handed Input in a Compound Task. Proceedings of the CHI'94 Conference on Human Factors in Computing Systems, 417-423.
9. Kabbash, P., MacKenzie, I.S. & Buxton, W. (1993). Human performance using computer input devices in the preferred and non-preferred hands. Proceedings of ACM InterCHI `93. Amsterdam, 474-481.
10. Kahneman, D., (1973) Attention and Effort, Englewood Cliffs, NJ, Prentice-Hall
11. Kantowitz, B., A., (1991) Effects of Response Symmetry upon Bi-Manual Rapid Aiming. Proceedings of the Human Factors Society 35th Annual Meeting. Human Factors Society, 1541-1545
12. Kirsh, D. and Maglio, P. (1994) On Distinguishing Epistemic from Pragmatic Action. Cognitive Science, Vol. 18. 513-549.
13. Leganchuk, A., Zhai, S.& Buxton, W. (in press). Manual and Cognitive Benefits of Two-Handed Input: An Experimental Study. To appear in Transactions on Human-Computer Interaction..
14. Oliviero, A., and Scarpetta, R., (1981) A new approach to contour coding. In Computer Graphics Image Processing. 15(1), 87-92.
Our experiment requires the definition of a distance function between curves that matches visual perception. This is not as straightforward as it may appear, as many commonly used functions, such as computing the area between the two curves, or taking the maximum of the minimum distances between each point of the two curves can exhibit a lot of cases in which two visibly different curves have a very close distance. Bartels [1] use a distance function defined by Oliviero and Scarpetta [14]. This distance is adequate for matching somewhat distant curves, but requires a few trial and errors to precisely adjust its parameters. Instead, we used our own distance function, extracted from [2]. We argue that this distance reflects well the perceptual notion of distance within curves. Finally, it is fast to compute.
Assuming that T, the target curve and M, the matching curve are arc-length
parametric function i.e. the parameter for a given curve varies in the range
[0..L] where L is the length of the curve. Assuming that d is a distance
between two points (in our case, d is the euclidean distance squared). Then
we define the distance between T and M by the following formula:
The minimum of the two values means that we don't care of the relative
directions in which the T and M curves were traced: M and T can be a good
match even though M was traced in the reverse direction of T. The integral
formula means that we compute the average value of all the distances between
individual points on each curve. When applied to a discrete context of curves
displayed on the screen, its distance is roughly equivalent to the number
of pixels by which the two curves differ over their whole length (i.e. choosing
a threshold of 10 means that there are about 10 pixels of difference between
the two curves over their whole length).
