Leganchuk, A., Zhai, S.& Buxton, W. (1998).Manual and Cognitive Benefits of Two-Handed Input: An Experimental Study. Transactions on Human-Computer Interaction, 5(4), 326-359.

Manual and Cognitive Benefits of Two-Handed Input:

An Experimental Study

Andrea Leganchuk, Shumin Zhai,and William Buxton

Input Research Group,
Computer Systems Research Institute,
University of Toronto,
Toronto, Ontario, Canada, M5S 1A4.
{andrea, willy, zhai}@dgp.toronto.edu.
Abstract

One of the recent trends in computer input is to utilize users’ natural bimanual motor skills. This paper further explores the potential benefits of such two-handed input. We have observed that bimanual manipulation may bring two types of advantages to human-computer interaction: manual and cognitive. Manual benefits come from increased time-motion efficiency, due to the twice as many degrees of freedom simultaneously available to the user. Cognitive benefits arise as a result of reducing the load of mentally composing and visualizing the task at an unnaturally low level imposed by traditional unimanual techniques.

Area sweeping was selected as our experimental task. It is representative of what one encounters, for example, when sweeping out the bounding box surrounding a set of objects in a graphics program. Such tasks can not be modelled by Fitts' Law alone (Fitts, 1954) and have not been previously studied in the literature.

In our experiments, two bimanual techniques were compared with the conventional one-handed GUI approach. Both bimanual techniques employed the two-handed "stretchy" technique first demonstrated by Krueger (1983). We also incorporated the "Toolglass" technique introduced by Bier, Stone, Pier, Buxton and DeRose (1993). Overall, the bimanual techniques resulted in significantly faster performance than the status quo one-handed technique, and these benefits increased with the difficulty of mentally visualizing the task, supporting our bimanual cognitive advantage hypothesis. There was no significant difference between the two bimanual techniques.

This study makes two types of contributions to the literature. First, practically we studied yet another class of transaction where significant benefits can be realized by applying bimanual techniques. Furthermore, we have done so using easily available commercial hardware in the context of a task common to graphical interaction. Second, theoretically this study contributes to our understanding of when and how to use bimanual interaction techniques in future systems and why bimanual interaction techniques they have an advantage over unimanual techniques. A literature review on two handed computer input and some of the most relevant bimanual human motor control studies is also included.

Categories and Subject Descriptors: H.1.2 [Models and Principles]: User/Machine Systems - human factors; H.5.2 [Information Interface and Presentation]: User Interfaces-input devices and strategies, interaction styles; I.3.6 [Computer Graphics]: Methodology and Techniques-interaction techniques;

General Terms: Human Factors, Experimentation, Design, Measurement.

Additional Key Words and Phrases: Input devices, Two-handed input, Bimanual input.


 

Introduction

Contemporary GUIs have come to be known as Direct Manipulation systems (Shneiderman, 1983). Correspondingly, the research described in this paper is based on the premise that one way to improve such systems is by extending the richness of what can be manipulated and the directness with which it can be done.

One approach to doing so is to develop interaction techniques that take advantage of the user’s potential to continuously coordinate both hands in the performance of a broad range of tasks. A number of systems have demonstrated how this potential can be employed in human-computer interaction (Buxton and Myers, 1986; Sachs, Roberts and Stoops, 1991; Bier et al., 1993; Hinckley, Pausch, Goble and Kassell, 1994; Kabbash, Buxton and Sellen, 1994). While these examples have demonstrated the effectiveness of such techniques, they have not yet penetrated into everyday applications. One reason for this may be the overall cost in equipping systems with input devices which afforded this class of interaction. However products such as, the multi-device sensing capability of the digitizing tablets from Wacom (Wacom, 1994), for example, address this — at least for graphical applications.

The other issue has to do with developing a better understanding of where, when and how best to use two-handed techniques, and why they work. It is therefore important to review, augment and consolidate the relevant literature. The remainder of this Introduction reviews some of the representative bimanual prototypes, and then discusses some pertinent experimental and theoretical studies in the literature. TWe then present the central hypothesis of the current work is then presented — the motor and cognitive benefits of two-handed input.

The representative prototypes of two-handed input

Two-handed input, per se, is not a new topic in user interface technology. For example, typing on a QWERTY keyboard uses two hands, but does not fall into the class of interaction studied in this paper.

Typing can be characterized as a coordinated bimanual activity where each hand is performing a series of discrete tasks. While one of the most effective ways of entering text into a computer, this does not reflect the more continuous nature of most everyday bimanual actions.

Sketchpad (Sutherland, 1963), is an early example of a system that not only used both hands, but did so in a way that went beyond the discrete types of actions seen in typing. Users drew directly on the computer screen with a light pen using one hand, and could modify the pen’s action with the press of a button using the other hand. The buttons were used in conjunction with the pen in this manner to create, move, copy or erase straight line or circular shapes. This is a very early example of a two-handed input interface where the left and right hands performed highly integrated actions.

Another early use of this type of two-handed input was in 1982 when the Xerox Star computer system was introduced (Smith, Irby, Kimball, Verplank and Harslem, 1982). In addition to the unique new graphical interface, the designers used a set of buttons on the keyboard to add functionality. These early ‘function-keys’ were positioned on the left side of the computer’s keyboard for easy access by the left hand, while the mouse was held and manipulated with the right hand.. The user could select an object with the mouse and then use one of the function keys to specify the action to be performed. For example, to move a file one would select the file’s icon, depress the "Move" button and then select its new location.

Video games are by far the most common example of two-handed input. A typical game controller has a set of buttons on one side of the pad and a joystick (or a rolling button or set of four cursor keys) on the other side. Typically the buttons launch bombs or cause the game character to jump while the joystick controls the character’s directional movement. The game character’s defense often consists of having to keep moving to avoid getting killed while at the same time the launching offensive attacks like dropping bombs. This requires skill and good coordination of the two hands.

The last three examples all have the similar property that one hand performs a discrete task (pushes buttons) while the other hand performs a continuous task (moves a pointing device). A significant difference of the interfaces is that some require the two hands to operate sequentially (Star) while others require coordination between the two (video games).

Krueger (1983) contended that users should be able to manipulate graphical objects like objects in the physical world. He demonstrated that this was possible for a range of tasks, using both hands to stretch, position and rotate boxes and other graphical objects in an intuitive and natural manner. This was perhaps the earliest example of coordinated and integrated bimanual interaction, where both hands were performing continuous, rather than discrete, tasks.

Sachs, Roberts and Stoops (1991) developed a two-handed system, 3-Draw, for 3D CAD applications. With their system, a user held a pair of six-degree-of-freedom input devices, mounted on a light hand-held tablet, or palette, and a stylus respectively. The palette corresponded to the base (construction plane) of a 3D object being created and the stylus was used as a drawing and editing tool. The designer manipulated the palette in the real world to rotate and translate the object in the virtual world. The stylus was held in a position relative to the palette and used to sketch the object’s curves.

Hinckley et al. (1994) designed a 3D two-handed input system using passive props. In order for neurosurgeons to specify a slice of the brain data to view, the user manipulated two props, a head prop and a cutting plane prop. The position and orientation of the props could be sensed by the computer. By placing the plastic sheet on the doll’s head, the surgeon could control what "slice" of the brain was displayed.

In 1993, Bier et al. developed a new bimanual interaction technique called Toolglass. This employed an interface widget that behaved as though it were a transparent sheet of glass upon which resides tools which can be used within an application The Toolglass sheet’s position was continuously controlled by the non-dominant hand, while the dominant hand controlled the system cursor. The Toolglass sheet lay between the cursor and the graphical objects on the "desktop." To use a tool on the Toolglass sheet, the cursor was positioned over the object to be affected, and the Toolglass sheet positioned such that the desired tool lay between the cursor and that object. One could then ‘click through’ the tool to cause it to affect the object below. Since the Toolglass is transparent, both the tools and the application behind can be seen at the same time.

Hinckley et al. (1994) designed a 3D two-handed input system using passive props. In order for neurosurgeons to specify a slice of the brain data to view, the user manipulated two props, a head prop and a cutting plane prop. The position and orientation of the props could be sensed by the computer. By placing the plastic sheet on the doll’s head, the surgeon could control what "slice" of the brain was displayed.

For the rest of this study, we shall be concerned with the general class of bimanual interaction introduced by Krueger and further developed by Bier et al. Sachs et al., and Hinckley et al., that is, systems where both hands work in a continuous, rather than discrete, manner, and where they are coordinated and integrated in the performance of a particular "direct manipulation" task.

Empirical studies and theories of two-handed input

While the list of bimanual prototypes is fairly long, the converse is true for the list of relevant empirical studies.

Dillon, Edey and Tombaugh (1990) conducted an experiment which examined performance in another compound task. They compared five interaction techniques for a task where the user would select from a menu and then draw a line elsewhere on the screen. Of the tested techniques, one was a standard GUI one-handed method where one cursor would be switching between menu selection and drawing in order to complete the task. They also included two variations of a "two cursor" method which employed the use of both hands, one controlling a cursor over a menu for selection and the other for drawing (the menu size varied for the two conditions). In this experiment a small but insignificant advantage of two-handed input was found over one-handed input.

In 1986 Buxton and Myers performed experiments which evaluated the benefits of two-handed input for compound tasks. They performed two experiments, one on a positioning-scaling task, the other on a navigation-selection task. Both experiments had the property that both hands were performing continuous tasks. In addition, the two hands could move either in parallel or sequentially while performing either experimental task.

The positioning-scaling experiment was designed such that the left hand scaled an object while the right hand positioned it. The results showed, without encouragement, all but one of the participantsubjects performed some of the task in parallel and the level of parallelism correlated to performance improvement. Because participantsubjects spontaneously and almost immediately exhibited the ability to perform both tasks in parallel, the experimenters concluded that no significant additional cognitive load was imposed by the utilization of two-handed input.

In the second experiment conducted, the left hand navigated through a document (controlled the scrolling) while the right hand selected designated pieces of text. This experiment was performed by both novice and expert users using both unimanual and bimanual techniques. Although little parallelism was exhibited, the two-handed technique was still significantly faster than the traditional one-handed technique. This was due to the fact that no switching was needed between scrolling and selecting in the two-handed technique — each hand being effectively in "home position" for its own particular task.

The two experiments by Buxton and Myers established the first empirical evidence that the use of two hands in human-computer interaction resulted in higher motor manipulation efficiency without, as explained above, imposing significant additional cognitive load.

Dillon, Edey and Tombaugh (1990) conducted an experiment which examined performance in another compound task. They compared five interaction techniques for a task where the user would select from a menu and then draw a line elsewhere on the screen. Of the tested techniques, one was a standard GUI one-handed method where one cursor would be switching between menu selection and drawing in order to complete the task. They also included two variations of a "two cursor" method which employed the use of both hands, one controlling a cursor over a menu for selection and the other for drawing (the menu size varied for the two conditions). In this experiment a small but insignificant advantage of two-handed input was found over one-handed input.

In contrasting the results of Buxton and Myers with those of Dillon, Edey and Tombaugh, we must conclude that the benefits of two-handed input are task and design dependent. Clearly a deeper level of understanding of human bimanual function is needed for the successful design of two-handed computer input.

Various human motor control theories on the relative performance of two hands have been proposed and tested (e.g. Flowers, 1975; Todor and Doane, 1978; Schmidt, Zelaznik, Hawkins, Frank and Quinn, 1979). Many of these studies on bimanual action have demonstrated the co-operative nature of the two. It is a common observation that it is difficult to rub ones stomach while tapping ones head. The difficulty lies in the independence of the two tasks. A common experimental paradigm used in two-handed bimanual control is production of polyrhythm: two conflicting but isochronous sequences. Most people have great difficulty in coordinating the two hands in such tasks in which the two rhythms are not integer multiples of each other. Only skilled musicians following extensive training are able to accurately perform complex polyrhythms with ratio of 5:3. Studies showed this was possible only when participants adopted an integrated motor organization in which movements of the slow hand were subordinate to movements of the fast hand (Summers and Kennedy, 1992). Furthermore, simultaneous independent timing for the two hands could not be achieved even when parallel control was encouraged by training the participants at producing the required tapping frequency with each hand independently first. The interdependence of bimanual movements has also been found in many other studies. For example, Peters (1985) showed that even for skilled participants, there were severe resource limitations in terms of providing two independent streams of timed signals for the initiation and termination of movement in the two hands.

However, tThe one that is most helpful bimanual action research in understanding the conflicting results seen in the two studies discussed above that can be directly applied to two handed computer input is Guiard’s kinematic chain (KC) theory (1987). According to this analogical model, the two hands function as serially assembled links, with the left (or non-dominant) hand as the base link and the right (or dominant) hand as the terminal link. Fundamental to this theory is the cooperative and asymmetric nature of the two hands. Four basic characteristics of the KC model are (Guiard, 1987; Guiard and Ferrand, 1996in press):

  1. Right-to-left spatial reference: the left hand sets the frame of reference for the action of the right hand.
  2. Left-Right scale differentiation: the granularity of action of the left hand is coarser than the right, the left hand movement is macrometric and the right hand movement is micrometric.
  3. Left-hand precedence in action: the sequence of motion is left followed by right.
  4. Right hand dominance: because the right hand is on the terminal (distal) end of the kinematic chain, human subjective preference between the two hands tends to be placed on the right hand side.
If one re-examines Buxton and Myers, and Dillon, Edey and Tombaugh in light of Guiard’s KC theory, an explanation for the different results emerges. What is seen is that Buxton and Myers’ experimental tasks conformed to Guiard’s four characteristics, while those of Dillon, Edey and Tombaugh did not. Thus, both an explanation for the difference in results, and some additional experimental support for the model are provided.

Guiard’s KC theory of bimanual function is also reflected in the design of the Toolglass technique (Bier et al., 1993), discussed earlier. Thus, we should expect it also to result in significant improvements in performance.

The technique was empirically evaluated in a study conducted by Kabbash, Buxton and Sellen (1994). Their experiment compared four different interaction techniques for the task of drawing colored line segments between dots. This task required a menu selection of the correct color and then a drag to create the line from one dot to the next. The techniques included:

  1. a one-handed technique with a standard tear off menu, a single cursor moves between selecting from the menu and drawing the line;
  2. a two cursor technique where one cursor (non-preferred hand) selected from the menu, and the other drew the line (similar to Dillon, Edey and Tombaugh, 1990);
  3. the Toolglass technique where the non-preferred hand positioned the Toolglass over the dot, and the other hand dragged through the appropriate color to the next dot; and
  4. a palette technique, similar to painter’s palette, where the non-preferred hand moves a standard tear-off menu such that movement time taken to move the single cursor controlled by the preferred hand between selecting from the menu and beginning the draw can be minimized.
Kabbash, Buxton and Sellen found the Toolglass technique, which conformed to the KC theory, to be superior over all the other techniques. First, it required fewerrequired the least number of motor operations. Second, the tools and drawing area were always in the same visual space, thus eliminating the possibility of divided attention. The experimenters noted that users were using their left hand the most in the Toolglass technique, yet this obviously did not hinder performance.

Finally, it is worth notingworthwhile to be reminded that the results also show that two hands are not always better than one. As in the Dillon, Edey and Tombaugh study, the two-cursor technique performed poorly — in this case (insignificantly) worse than the one-handed technique. So, while there are benefits to be gained, it is important that we undertake the research that will help us better understand how to obtain them.

Although most relevant to the current study, Guiard (1987) is but one approach to the study of human bimanual motor performance in the basic human motor control literature. Other studies on bimanual action tend to focus on elemental task experimentation. The co-operative nature of the two hands, however, was also be demonstrated at such a level. It is a common observation that it is difficult to rub ones stomach while tapping ones head. The difficulty lies in the independence of the two tasks. A common experimental paradigm used in two-handed bimanual control is production of polyrhythm: two conflicting but isochronous sequences. Most people have great difficulty in coordinating the two hands in such tasks in which the two rhythms are not integer multiples of each other. Only skilled musicians following extensive training are able to accurately perform complex polyrhythms with ratio of 5:3. Studies showed this was possible only when participantsubjects adopted an integrated motor organization in which movements of the slow hand were subordinate to movements of the fast hand (Summers and Kennedy, 1992). Furthermore, simultaneous independent timing for the two hands could not be achieved even when parallel control was encouraged by training the participantsubjects at producing the required tapping frequency with each hand independently first. The interdependence of bimanual movements has also been found in many other studies. For example, Peters (1985) showed that even for skilled participantsubjects, there were severe resource limitations in terms of providing two independent streams of timed signals for the initiation and termination of movement in the two hands.

Having reviewed the literature on two-handed computer input and some the most relevant bimanual human motor control studies, we now tend to the thesis of the current study, which offers a complimentary perspective to the existing views of bimanual computer input. While Guiard’s KC theory of bimanual action offers a helpful framework for designing two-handed input techniques, particularly on the asymmetrical division of labor of the two hands, it does not explain all the benefits of two handed input.We found that performance improvement with two-handed input often could not be explained at the motor-level alone. Part of it had to be attributed to cognitive factors.

Motor and cognitive advantage of two-handed input

Studies such as Buxton & Myers and Kabbash, Buxton & Sellen show that improved time-motion efficiency can be achieved using appropriate bimanual techniques. However, can the benefits be attributed simply to improved motor-sensory efficiency, or are there cognitive reasons as well?

One of the basic tenets of the acquisition of cognitive skill is the notion of chunking (Simon, 1974). Anderson (1980; 1982), for example, describes the acquisition of cognitive skills as based upon the compilation and proceduralization of knowledge into chunks formed around underlying sub-tasks. Likewise, Card, Moran & Newell (1980) describe users’ methods in the Keystroke-Level Model as being

…composed of highly integrated submethods ("subroutines") that show up over and over again in different methods. We will call them method chunks, or just chunks. There are numerous example of chunking in our daily life, such as remembering long (telephone)numbers as a few groups of digits, rather than individual digits. If chunking plays such an important role in skills and their acquisition, then a reasonable strategy to facilitate their acquisition is to design the system’s motor-sensory affordances to reinforce the desired chunking structures. Design with the appropriate compatibility (Fitts & Seeger, 1953; John, Rosenbloom & Newell, 1985) is one mechanism for achieving this, and is an approach explored by Rosenbloom (1985). It is also the path followed by Buxton (1986).

Applying the notion of chunking and phrasing to (graphical) manipulation at the computer interface, Buxton argued that the physical affordances at the motor level of interaction could be designed to be compatible with and reinforce the desired chunking of sub-tasks at the cognitive level. He suggested that there were two types of "glue" that could be used as the catalyst to this aggregation: muscular tension and continuity of motion ("kinesthetic continuity"). Simply stated, he argued task elements that should be conceptually chunked should be physically chunked into a single gesture, or "phrase" (where, like in music, tension and movement are the key elements of establishing structure).

A common example of chunking is seen in making a selection using a pop-up menu in a GUI. Another example is illustrated in the common proof-reader’s symbol for ‘move’. In both cases, a number of sub-tasks are "chunked" into a single aggregate, or "phrase." (And, as in music, the phrase holds the elements together into a connected flow which is not generally interrupted.) In the case of the pop-up menu, muscular tension of holding down the mouse button is the glue that binds the elements of the phrase, whereas in the proof-reader’s move symbol, the bond is the kinesthetic continuity associated with articulating the mark of the symbol. While syntactically rich, due to the phrasing, each of the example transactions is perceived as a single gestalt.

From this perspective, we can think of the acquisition of cognitive skill as relating to the compatibility between motor action and aggregation of cognitive unit tasks. Conversely, one could argue that some problems in skill acquisition are a consequence of a lack of compatibility with the pragmatics of how actions are articulated. This was the perspective from which we investigated the potential cognitive benefits of bimanual input. Our basic hypothesis was that appropriately designed bimanual techniques should chunk better at the motor-level, and therefore be more compatible with the mental model of the user, as acquired from a lifetime of living in the everyday world.

Figure 1: Using traditional unimanual techniques, moving Rectangle A to Rectangle B requires (multiple) iterations of translation, rotation, and scaling. In contrast, using a bimanual technique that assigns two hands to controlling two opposing vertices, all of the three aspects can be "chunked" into one integrated process, which is closer to how one naturally views such a task.
More specifically, using traditional unimanual input techniques, many graphical manipulation tasks, such as to move, scale and rotate an object (e.g. a rectangle), are forced to be executed on an elemental level (one aspect at a time with iterations), which impose unnecessary cognitive burden on the user (See Figure 1). Conversely, two-handed input may allow the user to perform at a natural level of chunking that corresponds to daily activities, possibly resulting in the following interrelated cognitive advantages:
  1. Reduce and externalize the load of planning/visualization in unimanual input. Because two-handed input allows the user to treat the task as a larger and more natural gestalt, the user no longer needs to compose, "think" and plan the elemental steps of a task.
  2. Rapid feedback of manipulation results in a higher level of task: the user immediately sees the result of action in relation to the goal state.
  3. Support epistemic action: the user may take advantage of the two-handed input and perform actions of an epistemic nature in addition to those of a pragmatic nature. (Kirsh and Maglio, 1994).
Together with our colleagues (Owen, Kurtenbach, Fitzmaurice, Baudel and Buxton, 1996), we have made the above observations in a number of two-handed tasks. In this paper, we present the results of an empirical study that indicate the cognitive advantages of two-handed input, in addition to the physical motor efficiencies that have been previously found in the literature.

Experiment 1

Task Selection

We had three criteria in mind when we chose the task for our experimental study. First, we wanted to select tasks that are practical in real computer applications so that the results of our experimental study can be applied to designing practical interaction techniques. Second, we wanted to choose a task that has not been extensively studied in the HCI literature so that the empirical findings in regards to two-handed input can be established on a broader base. Third, we intended to select a task that has the testing power to accept or reject our hypotheses (discussed in the section, The Hypotheses).

Studies in computer input have concentrated on "point-click" type selection tasks. This type of selection can generally be modelled by Fitts' Law (Fitts, 1954), and is fairly well understood. A form of graphical selection which has seen widespread use, yet is far less studied, is the sweeping of a bounding box around an area or group of objects. Figure 2 shows an example where the user is attempting to place a minimum bounding box around just the woman in the picture. This issue arises in both computer drawing packages as well as painting programs. Users often must select accurately a specific area of their drawing so as to include some parts in their selection while excluding other parts. This proves to be a difficult task when these parts lie close to one another but distant from the location of the beginning of the drag.. This class of transaction is very common and is representative of a larger class of graphical interactions. It has not been previously studied yet has been observed to be highly prone to user error.

Figure 2:Generalized example of the bounding box issue. The user is currently dragging out a selection box. The initial mouse press at the top left was incorrect (they are including part of the man’s arm unintentionally).
The experimental task used in our study is an abstraction of the above area sweeping task. The participantsubjects were asked to minimally enclose the object presented with another figure. Six different basic geometric forms were chosen for presentation: squares, rectangles, circles, ovals, equilateral triangles and isosceles triangles. Two shapes were used as the encompassing figure, a rectangle and an oval.

ParticipantSubjects were directed to use an oval to enclose triangles, squares and rectangles. Similarly they were directed to use a rectangle to enclose circles and ovals. Figure 3 illustrates some examples of correct stimulus-response (i.e. target figure-encompassing figure) pairs.

Figure 3: Examples of minimal enclosures. The solid objects represent the target figures. The dotted shapes represent the user’s encompassing response figure.
The user was required to draw the encompassing figure (their response) so that it matched the stimulus’ boundary to within one screen pixel. Feedback was provided in the form of red dots at matching stimulus-response points once the user released the encompassing figure. For a square or rectangular stimulus, the encompassing oval would have to match within one pixel at four points. For an oval or circle stimulus, the encompassing rectangle would have to match within one pixel at four points. For a triangle stimulus, the encompassing oval would have to match within one pixel at three points. Refer to Figure 4 for a sample of where the red dots would appear.

Each encompassing figure has four control points, represented by the black square handles, which appear when the input device is released. The control points are points which by moving them, one can change the shape of the resulting figure.

Once the encompassing figure was drawn, if it did not meet criteria, a one screen pixel maximum distance between stimulus and response edges, then the user would have the opportunity to readjust the figure until it matched by reacquiring any of the four control points.

If the user missed the handle, the handles disappeared, the figure became ‘de-selected.’ The user re-selected the encompassing figure by clicking anywhere within the figure.

If the user chose the wrong encompassing figure tool (such as the rectangle tool when presented with a square) the system beeped and would not activate the tool. The user would then attempt again to choose the correct tool.

The user performed the task with a Wacom Tablet holding a stylus in their right hand and a 4-button puck in their left. This apparatus is described in detail in the section to follow entitled The Apparatus.

The Techniques

In this experiment we study three techniques using the same hardware framework. The only differences presented to the user will be in the form of the technique’s specific properties as outlined below.

Conventional One-handed Technique (C1)

This unimanual technique was modelled on the techniques and methods currently employed in traditional graphical user interfaces (GUIs).

The tools (encompassing figures) are available on a tearoff palette which may be moved by dragging its header frame to a new location. A tool is chosen by clicking on the palette on the desired tool. The tool will highlight to show it has been selected and the cursor, when over the drawing surface, will change from the default arrow to a drawing cursor. The user may then drag on the drawing surface to use their chosen tool to create the encompassing figure.

Upon release, the standard handles found in most drawing applications appear at the four control points of the encompassing figure. Adjustments may then be made to the figure by dragging these handles to new locations. See Figure 4 for an example of an encompassing figure with handles.

Figure 4: Example of encompassing figure (dotted figure) with handles (black small boxes at corners).
Stretching with Two Hands (S2)

This technique takes advantage of using both hands to manipulate the encompassing figure based on a technique initially introduced by Krueger (1983). The tool is selected from a standard GUI tearoff menu as described above for the unimanual technique. The tool will highlight to show it has been selected and the cursor, when over the drawing surface, will change from the default arrow to a drawing cursor.

When the user presses down on the drawing surface the location and size of the encompassing figure is controlled by both hands. A cursor appears at opposing corners of the figure, indicating the points from which the left and right hands can "stretch" it. The left and right hand cursors are both stylized arrows but are oriented differently to differentiate between them. This is illustrated in Figure 5. Using this technique, the user can simultaneously scale and position the figure in a manner which strongly mimics how similar transactions are performed in the everyday world.


Figure 5: Using two hands to stretch a square.

ParticipantSubjects may reacquire the encompassing figure to make adjustments by dragging a handle with the right cursor as in the unimanual technique. The right hand is attached at the control point that was pressed and the left hand is attached to the control point at the opposite corner of the figure. The figure may then be manipulated using both hands.

Toolglass with Two Hands (T2)

This condition integrates the previously described two-handed stretching method with the Toolglass technique introduced by Bier et al. (1993). Tool selection is effected using a "click-through tool," using the right hand to press through a transparent tool palette whose location is controlled by the left hand. What is novel compared to previous click-through tools reported in the literature is that once the tool is selected, the tool palette disappears, and the "other" hand now controls the position of one corner of the encompassing figure instead of the position of the palette. The right hand controls the opposing corner. The users may then size and position their encompassing figure accordingly, the same as in the two-handed stretching technique. Once they release the input device, the palette reappears at their left hand. The participantsubject may reacquire and manipulate the figure in the same manner as the two-handed stretching technique.

In all three techniques the size of the palette menu was kept the same to normalize across experimental conditions.

The Hypotheses

Our analysis began with the recognition that this class of interaction is a compound task. The steps, as implemented in conventional one-handed technique (C1), are as follows:

  1. select the encompassing figure to be used using the tool palette;
  2. specify (commit) the position of the upper left corner of the encompassing figure by positioning the cursor and dragging;
  3. sweep out the area from that corner to the lower right corner;
  4. specify (commit) the position of the lower right corner of the encompassing figure by releasing the button press.
Note that the encompassing figure is therefore defined sequentially by two points, the initial drag starting point and where the drag is released.

Frequently, the user must repeat steps 2 to 4 of the aforementioned list since typically either the vertical or horizontal alignment of the first specified control point is incorrect (as also demonstrated previously in Figure 2). An analysis of this simplified characterization of the task is shown in Figure 6.

Figure 6: Characteristics of enclosing a target using a conventional one handed technique (C1).
Figure 6 contains a Tension vs. Time graph which is intended to show the phrasing of the task. An increase in Tension represents the user pressing down with their input device. A decrease in Tension represents the release of the input device. Constant tension represents the articulation of a drag.

The first phrase stands for the selection of the tool. The onset of the second phrase represents the selection of the first control point (press), and the trailing edge of the phrase, the specification of the second control point (release). These two control points define the response’s position and size. The dotted additional phrases represent potential phrases that may be required to correct the placement of the stimulus corners to complete the task.

What Figure 6 also shows is at what stage in the process the user commits to the specification of key parts of the transaction. When the user initially presses down to begin drawing their selection, they have committed to the placement of that ‘half’ of the figure. That control point cannot be moved until the drag is released and the point is reacquired by pressing on the handle at that control point. From this, we can derive the likely cause of potential error, namely that the specification of the first control point must be made before there is any feedback from the transaction to aid in its proper alignment.

Recognizing the potential error and its cause (insufficient feedback at the time of commitment), there are two strategies to pursue. We could add the feedback at the time. This could be done with something like cursor cross hairs which extend to the edges of the screen. While such a strategy can be useful for the tasks concerned in this study, displaying large full screen cross hair cursor can be distracting in other tasks. An alternative is to defer commitment until the feedback exists. The two new techniques that we used in this study chose the second, more general strategy.

The "Stretching with Two Hands" (S2) technique’s phrasing is illustrated in Figure 7. Again there are two phrases. Tool selection is as with the C1 case, a single click. The difference lies in the fact that while sweeping the area encompassing the presented stimulus, the initial corner is not permanently "fixed" as it was in the C1 case. The user may simultaneously move the two defining control points of the figure, that which is attached to their left hand and that which is attached to their right hand.


Figure 7: Characteristics of enclosing a target using the "Stretching with Two Hands" technique (S2).

What this means is that one is not committed until the end of the second phrase, when they release their drag. The user sees the results, where their encompassing figure lies, before committing to the final placement of either control point.

The "Toolglass with Two Hands" (T2) technique is characterized in Figure 8. In this case, selection of the tool and sweeping out of the area are integrated into a single phrase. After tool selection, the user may manipulate the encompassing figure with both hands, as in the S2 technique.

Figure 8: Characteristics of enclosing a target using the Toolglass (TG2) technique.
With both two-handed techniques, the user can manipulate the size and location of the encompassing figure at the same time, integrating the sweeping task into one "gestalt." In contrast, with the one-handed technique, the user is forced to manipulate one aspect of the task at a time and commit to it by guessing or mentally visualizing the final effect of that commitment. In the case of the two-handed techniques, this cognitive visualization process is externalized and reduced.

With the one-handed technique, the "cognitive load" of the visualization process varies with the size and shape of the encompassing figures. For example, in the case of encompassing an oval with a rectangle, the user must visualize where to initially press down to begin their sweep. This choice is made by mentally projecting lines from the edges of the oval and determining the point where they intersect (as demonstrated in Figure 9, the user must visualize initial point I). As the distances from point a to point b as well as from point c to point d increase (i.e. as the oval increases in size), it becomes harder and harder to reliably mentally visualize the extension lines and accurately pick point I. In other words, the larger the target figure, the higher the cognitive load of visualization load the unimanual technique imposes to the user.

Figure 9: With one handed input, the user must mentally visualize the dotted lines, extending them from point a to point b and from point c to point d. They must determine where these lines intersect and press down at the point marked I to begin their sweep to create the encompassing rectangle. Note that the length of these dotted lines is proportional to the size of the target.The larger the oval is, the higher a cognitive load of visualization it imposes on the user to accurately project point I.
After pressing down at point I, the user proceeds to drag to completion point C. Even if the pick of point I is not accurate, the user can ensure the release of the input device at point C is accurate because the edges of the bounding rectangle are visible during the sweep and thus their intersection with the oval is visually verifiable. The user may then travel back to point I and adjust it if it was in error, and this adjustment can also be made accurately because the edges of the bounding box are again still visible thereby giving adequate feedback which was not there at the outset. Therefore, this method will probably require one adjustment of point I to achieve criteria.

The case where the user is required to encompass a rectangle with an oval is more cognitively demanding. As in typical graphical drawing packages, the oval is specified by sweeping out a rectangle into which the oval is drawn. Thus, in order to correctly encompass a rectangle with an oval the user must mentally visualize where the oval must be and then mentally visualize the box which defines that oval, which means visualizing extension lines from the edges of that oval (similar to the visualization discussed in Figure 9) to determine the initial press down point I (visualization graphically depicted in Figure 10). If this guess of point I is incorrect the user can still sweep out the oval, but in contrast to the previous example, the release at point C is not visually verifiable. This is due to the fact that subsequent adjustments of point I will affect the overall placement of the oval. Thus the user may be forced to travel repeated times between their initial point I and the completion point C in order to achieve criteria. Again, these effects will be enhanced by size.

Figure 10:A user must visualize the oval (in graey) which encompasses the rectangle then they must visualize the extension lines and pick the points marked I as well as C accurately.
Finally, for the case of encompassing a triangle with the oval, the cognitive burden will lie somewhere between the previous two cases. The triangle has an advantage over the previous case in that the visualization of the vertical alignment of the first point (I) can be projected from the triangle’s point (Figure 11). It is only for the horizontal alignment that one must visualize the encompassing oval. Therefore, the user has a slightly better chance at guessing point I fairly close to it’s actual location.

Figure 11:User must visualize the oval in order to find horizontal alignment of I, but must only project from triangle top point to find vertical alignment of I. In short, in addition to the motor inefficiency of switching between two control points with one cursor, the traditional one handed input also imposes a cognitive load of visualizing the correct control points. Such cognitive load increases as the size of the target figure increases and some shapes (such as a rectangle, to be encompassed by an oval) imposed a higher load than others (such as an oval to be encompassed by a rectangle).

With two handed input techniques, the user does not have to switch between the two control points and hence gain motor efficiency advantage. Furthermore, the need to mentally visualize the final locations of the lower than natural level of elements (control points) is also eliminated (externalized). The user hence also gains a cognitive advantage.

Therefore, working from these analyses, we established the following predictions.:

Based on the previous evidence of two-handed advantage (Buxton and Myers, 1986; Kabbash, Buxton and Sellen, 1994), albeit with a new class of tasks, we predicted:
 

Hypothesis 1:  Both two-handed techniques (S2 and T2) will outperform the conventional unimanual technique (C1).
According to our notion of chunking and phrasing, which expresses how efficient interactions are often marked by ‘chunking’ task elements into greater units (Buxton, 1986). In the current task, the unimanual technique (C1) afforded the user to operate at the lowest level of chunking of the three techniques. Different components of the task (tool selection, sweeping, and multiple resizing) had to be carried out by multiple, separated motor tension "chunks" (press and release cycles, see Figure 6). The stretching with two hands (S2) technique, integrated the multiple resizing cycles into one continuous motor tension chunk, but still kept tool selection as a separate unit (Figure 7). The tool glass technique (T2) integrated the entire task, from tool selection to area sweeping into one motor tension cycle (Figure 8). We hence expected:
 
Hypothesis 2: Performance will improve with level of integration (chunking), from C1 to S2 and from S2 to T2.
As complexity increases, it becomes ever harder to mentally position the encompassing figure’s corners. In the absence of feedback, it becomes increasingly difficult to visualize the task. Such a visualization process is cognitive, and that it will play a significant role in participants’ performance. Such an analysis predicted:
     
    Hypothesis 3: Performance difference between the one-handed technique and the two two-handed techniques will become more pronounced as more mental visualization or planning is required, i.e. as cognitive difficulty of the task increases.
We base our first hypothesis on the previous evidence of two-handed advantage (Buxton and Myers, 1986; Kabbash, Buxton and Sellen, 1994), albeit with a new class of tasks. Hypothesis 2 is based upon our notion of chunking and phrasing, which expresses how efficient interactions are often marked by ‘chunking’ task elements into greater units (Buxton, 1986). Hypothesis 3 is motivated by the observation that as complexity increases, it becomes ever harder to mentally position the encompassing figure’s corners. In the absence of feedback, it becomes increasingly difficult to visualize the task. Such a visualization process is cognitive, and that it will play a significant role in participantsubjects’ performance.

The Apparatus

The experiment was performed on a Macintosh Power PC 6100/60 AV model with a 17-inch colour monitor with 832 by 624 pixel resolution, using a Wacom UD-1212R tablet (Wacom, 1994) with "Multimode" capabilities (the tablet can sense two different devices simultaneously on its surface).

ParticipantSubjects held a Wacom 4-button ‘puck’ in their left hand and a ‘stylus’ in their right hand. For the one-handed technique, only the stylus was used. These two devices are shown in Figure 12, held on the tablet.

Figure 12: The Wacom stylus and puck on the Wacom tablet.
The puck is simply a ‘mouse-sized’ box with buttons on the top. It’s "hot-point" (where the screen cursor tracks it) is defined by the junction of the cross hairs in the circular see-through area at the top of the device. The puck lies on the tablet surface, is held like a mouse and glides over the surface of the tablet.

The stylus’ "hot-point" is its tip. To move the screen cursor with the stylus, users held the stylus like a pen and moved it slightly above the surface of the tablet (0-5 mm). If it was lifted too far from the surface of the tablet the cursor would freeze. It would resume movement once the device was brought close enough to be sensed again. Pressing down with the tip of the stylus was recognized as equivalent to a mouse button press.

The stylus and puck are untethered. For this experiment, none of the buttons on the puck were used, nor the barrel switch on the side of the stylus. The hot-point of the puck was also offset such that the screen cursor appeared to the right of the physical device to reduce physical collisions of the puck and stylus on the tablet surface.

The tablet was scaled such that a 21.7 centimeters (across) by 17.7 centimeters (down) area on its surface mapped to the complete computer monitor.

The ParticipantSubjects

Fifteen peoplesubjects (Ss) participated in this experiment, 12 males and 3 females. All participantsSs were right-handed. None had significant prior experience using an absolute positioning device such as the tablet, but most were very experienced computer users (use one every day). Most were novice users of computer drawing packages (use one 0-25% of their computer experience).

The Design and Procedure

Each participant subject performed all three techniques in a within-subjects design. The experiment was a 3 x 6 x 4 factorial experiment (Technique x Figure x Size). Technique order was counterbalanced using a Latin Square design: 5 participantsSs performed in the order of C1 S2 T2, 5 in S2 T2 C1 and 5 in T2, C1 S2.

ParticipantsSs were first instructed verbally, outlining the general requirements of the task. Before attempting each technique, they received verbal instructions outlining the properties of the technique and one practice block consisting of 12 trials with random shape, size and length to width ratio.

After practicing, participantsSs performed 2 blocks of trials. Each block consisted of 24 trials presented in a random order. The presented figure in each trial would be one of six shapes (Square, Rectangle, Circle, Oval, Equilateral Triangle, and Isosceles Triangle) and four discrete sizes (50, 130, 200 and 280 pixels). Each combination of size and shape would be seen once during the block totaling to the 24 trials in a block (6x4). The ratio of length to width for asymmetric shapes without rotational symmetry was constant for all at 1:2.

ParticipantsSs took approximately one hour to complete 2 blocks for each of the three techniques. After completing all three techniques participantsubjects were administered a questionnaire to elicit subjective feedback.

The Results

Total trial time was calculated as the time from selection of the tool to completion of the task. Overall, total trial time differed very significantly amongst techniques (F2,24 = 28.22, p < 0.0001). Table 1 shows the mean total trial time for the three techniques. We can see that two-handed techniques on the average save about 17% of the time it takes to do the same task using the conventional one handed technique (C1).
 
Mean Total Time
Standard Deviation
Standard Error
Technique
(seconds)
(seconds)
(seconds)
C1
14.74
8.16
0.3
S2
12.24
7.26
0.27
T2
11.61
8.61
0.32

Table 1: Mean total trial completion time for Experiment 1.


Pair-wise contrast tests for differences between means showed that both two-handed techniques were significantly better than C1 (p < 0.0001), but that the two two-handed techniques (S2 and T2) were not significantly different from each other (p = 0.16).

Subjectively, when asked which technique they liked best, users preferred S2 (stretching with two hands) over T2 (Toolglass with two hands), 8 to 6 with one user preferring C1. When questioned about which technique they liked the least, 10 users expressed dislike for C1, and 5 users expressed dislike of T2. None of the users chose S2 as their least liked technique.

Measurements of the left and right locations were recorded every third of a second which enabled us to calculate rough speed estimations for the left and right hands. The speeds of the left hand tended to be slower than the speeds of the right. A sample is shown in Figure 13 from a participantsubject that accomplished the task in one attempt (no need to reacquire the control points) using the S2 technique. A sample of one-handed trial (the C1 technique) is shown in Figure 14 from a participantsubject who took four attempts to achieve task completion (had to reacquire four control points before getting the encompassing figure correct).

Figure 13: Speed graphs of the left and right hands of a participantsubject that took one attempt to complete the task using the S2TH technique.
Evidences of Cognitive Benefits in Two-Handed Input

Thus far, the results have indicted that two-handed techniques were superior overto the one-handed technique for practical purposes. Theoretically, the performance difference could be a result of either physical efficiency or cognitive characteristics of the techniques. One obvious physical motion difference between one-handed and two-handed techniques lies in the (re)acquisition of the control points (handles). With the one-handed technique, the user has to switch between two control handles (Figure 14). Each reacquisition of a control handle is a Fitts’ law task. This alone could account for the entire performance difference between the one-handed vs. the two-handed techniques. It is therefore informative to remove the control handle reacquisition times from all the conditions, although it is an inherent part of one-handed manipulation.

Figure 14: Speed of the right hand of a participantsubject who takes 4 attempts to complete the task using the C1GUI technique.
Control point reacquisition time was defined as the time from a button up to button down. Note that a small percentage of the two-handed trials also required control point reacquisition, because the encompassing figure placement did not meet the accuracy criteria when the control points were released. However, this did not require the cursor to travel from control point to control point thus did not increase trial completion time significantly. The measures after removing control point reacquisition time are biased against the two-handed techniques because a participantsubject’s cognitive process is not necessarily halted during the reacquisition of control points.

After removing the control point reacquisition parts of the task, the one-handed and two-handed techniques became equivalent in terms of amount of motion needed to complete the task. The cause of performance difference between one-handed and two-handed after the removal could still be twofold: the physical motor efficiency difference due to the parallelism in the two-handed conditions and the cognitive visualization difference between the one-handed and the two-handed conditions. However, as analyzed earlier, two factors, size and shape of the targets, changed the amount of mental visualization needed. They therefore could serve as indicators of the importance of the cognitive factors in the performance differences between conditions.

The results after the removal of control point reacquisition times are shown in Table 2. After the removal, we see no significant mean performance difference between the one-handed technique and two-handed techniques. However, there was a significant interaction between technique and size (F6,24 = 3.34, p < 0.01).
 
Mean Total Time
Standard Deviation
Standard Error
Technique
(seconds)
(seconds)
(seconds)
C1
10.43
5.16
0.19
S2
10.66
5.26
0.2
T2
9.92
5.69
0.21

Table 2:Mean total trial completion time after removal of reacquisition time for Experiment 1.

As shown in Figure 15b (performance time after reacquisition time was removed), for the largest target whose final correct control points were the most difficult to visualize, two-handed techniques were significantly better than one-handed (p < 0.05 for C1 vs. S2 and p < 0.05 for C1 vs. T2). For the smallest target that did not require much planning and visualization, two-handed was in fact inferior to one-handed (p < 0.001 C1 vs. S2) or was not statistically different from the one-handed technique (p = 0.84 C1 vs. T2). If the advantage of the two-handed conditions were all due to the parallelism in the physical actions, we would expect at least equivalent performance to the one-handed for the smallest targets. Clearly the cognitive factor is an important contributing component to the performance difference between the one and two-handed techniques.

(a) 

(b) 

Figure 15: Interaction of Size and Technique with 95% confidence error bars. (a) total trial completion time (b) total trial completion time less control point reacquisition time
Figure 16b shows the results after the removal of control point reacquisition time in relation to the shape of the targets. We can see for the cognitively more demanding shapes, such as the square, two-handed techniques were still significantly better than one-handed (p < 0.0001 for both C1 vs. S2 and C1 vs. T2). For cognitively less demanding shapes, such as the circle, there was no significant difference (p = 0.17 for S2 vs. C1, p = 0.81 for T2 vs. C1). This again shows that the advantage of the two-handed techniques is both physical and cognitive.
(a)

(b)

Figure 16: Interaction of Shape and Technique with 95% confidence error bars. (a) total trial completion time (b) total trial completion time less control point reacquisition time
Combining these two analyses (in relation to size and shape of the targets), we can conclude that the advantages of two-handed techniques found in this experiment suggest more than physical efficiency. We observed the trend that as the size increased or the shape became more difficult the two-handed techniques would outperform the one-handed technique to a greater extent. In other words, for high cognitive load visualization load tasks (bigger size, difficult shape), we see an increased advantage with two-handed over one-handed techniques, even if the major time-motion factor is removed from the data. Such trends were statistically significant both before and after control point reacquisition time removal.

As cautioned by some experimental psychologists (Poulton, 1966; Poulton, 1989), a common pitfall of within-participantsubjects design of an experiment is the asymmetrical skill transfer effect. That is, with a within-participantsubjects design, the actual skill transfer from one condition to another might not be symmetrical, even though participantsubjects’ exposures to the conditions are equalized. This appears to be true of this experiment. The interaction of technique and order was shown to be a very significant factor (F4, 24 = 11.65, p < 0.0001).

The ordering effect did not significantly influence the trend that as size increased, the two-handed techniques outperformed the one-handed technique to a greater extent. The three way interaction of Technique x Size x Order was insignificant both before (F12, 72 = 0.77, p = 0.68) and after (F12, 72 = 0.79, p = 0.66) the control points reacquisition time removal. Likewise, Technique x Shape x Order was also insignificant both before (F20,120 = 1.32, p = 0.18) and after (F20,120 = 1.33, p = 0.17) the control points reacquisition time removal. This shows that the evidence of cognitive benefits in the two-handed conditions still hold regardless of the asymmetrical transfer effect.

We believe the asymmetrical skill transfer is related to the fact that the users did not have enough time to learn how to use the tablet (recall none had significant prior experience). This called for a follow-up experiment that would give users more time to learn the input devices as well as the techniques. In addition, following the evidence of cognitive benefits in two-handed interaction, we decided to use only the cognitively more demanding targets (large sized square and triangle shapes). We also changed the feedback design so as to give more rapid feedback as discussed in the following description of Experiment 2’s Task. It was observed during the first experiment that users frequently missed the target figure by a pixel or two when they thought they were accurate. The user would not know they missed the target until they released the input device. This delayed feedback might hinder one of the cognitive advantages of two-handed input (rapid feedback towards goal state).

Experiment 2

A follow-up experiment was conduct mainly for the purpose of clarifying if asymmetrical skill would still be a significant factor if more practice was given to each technique tested. In addition, the feedback method that might have hindered the task performance was also modified.

The Task

As mentioned, more rapid feedback was provided in this experiment. This was done such that when the user’s encompassing figure arrived at the correct spot, the red highlights would appear, regardless of whether the drag had been released. Once the drag was released, the figure was evaluated again and if all points still matched the system would record the trial as complete.

The Techniques

In the first experiment there was no significant difference between the performance of the two bimanual techniques. We chose to study only one of the two bimanual techniques, the "Stretching with Two Hands" (S2) technique. This technique was chosen over the Toolglass technique (T2) because no further improvement was found with the T2 technique, which integrates menu selection into two handed input. We also noted in the first experiment that the T2 technique was disliked by most participants.

The Apparatus

We used the identical experimental setup as for the first experiment.

The ParticipantSubjects

Eight peoplesubjects participated in this experiment, 6 males and 2 females. All participantsubjects were right handed and had not participated in any previous two-handed experiment. Only one participantsubject had significant prior experience using a tablet. All participantsubjects were experienced computer users (used one every day).

The Design and Procedure

We restricted this experiment to the shapes and sizes which are previously analyzed as cognitively more demanding. Only the square, rectangle, equilateral triangle and isosceles triangle were used for presentation shapes. Therefore the encompassing figure was limited to an oval shape. ParticipantSubjects were still required to choose the oval tool to start each trial. A random set of sizes, +/- 30 pixels of the largest size used in the first experiment (280 pixels), was generated and presented to all participantsubjects. The ratio for asymmetric shapes without rotational symmetry was constant for all at 1:2.

The overall reduction of experimental conditions (the number of techniques as well as reduced set of stimuli) allowed us to give more time for participantsubjects to perform each technique with each type of target. In addition, we also reduced our block size to 12 trials per block.

ParticipantSubjects were first instructed verbally, outlining the general requirements of the task. Before attempting each technique, participantsubjects received verbal instructions outlining the properties of the technique and completed one practice block consisting of 12 trials with random shape, size and length to width ratio.

All participantsubjects were able to complete 5 blocks of each technique in less than one hour. ParticipantSubjects were administered a questionnaire after completion of all trials to elicit subjective feedback.

The Hypotheses

The order effect probably confounded the main effect (performance difference between techniques) in the first experiment. Such an order effect might reduce to be insignificant, as more experience was given to each technique in this experiment. The selection of the size and the shape was intended to be more cognitively demanding for unimanual techniques and the design of instantaneous feedback of being on target was intended to facilitate the visualization process (another cognitive factor). With these changes, we hypothesized that the two-handed condition would outperform the one-handed condition to a possibly greater extent in this experiment than in Experiment 1.

Results

Total trial completion time differed significantly between the two techniques (F1,6 = 89.08, p < 0.0001). Table 3 shows the mean total trial time for each technique. The two-handed technique resulted in time savings of about 39%.
 
Mean Total Time
Standard Deviation
Standard Error
Technique
(seconds)
(seconds)
(seconds)
C1
15.84
7.18
0.33
S2
9.60
5.20
0.24

Table 3: Mean total trial completion times for Experiment 2.

In comparison to the results from the first experiment, the mean completion time using the one-handed technique has increased about one second for this experiment while the mean completion time using two-handed technique total time has decreased by over two seconds. Recall that in this second experiment we used only the more cognitively demanding targets (larger sizes and difficult shapes). We also provided rapid feedback and more practice with both techniques.

The interaction of technique and order is not significant (F1,6 = 9.64, p = 0.77), suggesting the asymmetrical skill transfer is no longer a significant effect. We therefore conclude that the differenent characteristicsce between the two techniques is stronger than the inherent instead of a by-product of the skill transfer transfer effect.

Learning significantly improved both techniques (p < 0.005) and the two-handed technique (S2) was consistently better than C1 for all blocks (Figure 17). The interaction of technique and block was not significant (p = 0.27). We hence do not expect further practice will change the results of this experiment.

Figure 17: Two and one handed technique total trial completion time in relation to experimental blocks with 95% confidence error bars.
The interaction of technique and shape on total trial completion time was not significant (p = 0.13) (Figure 18a). As discussed earlier, some shape of target are more difficult for the unimanual (C1) technique, due the greater difficulty of visualizing the correct control point locations. On the other hand, such shape difference should not affect two handed techniques, since the visualization process is externalized. Indeed, Ppairwise contrast tests revealed no significant differences among means of different shapes with S2 but C1 varied significantly with shape (Figure 18a).. In particular, Square vs. Isosceles Triangle, Square vs. Rectangle and Square vs. Equilateral Triangle were significantly different (p < 0.01) with the C1 technique

Again we removed the control point reacquisition time from the total completion times in order to remove the time-motion disadvantage of the one-handed technique. Technique was still significant (F1,6 = 25.46, p < 0.005). Figure 18b shows the results of our analysis in relation to the shape of the targets. For all shaped targets, S2 was faster than C1 (p < 0.01).

(a)

(b)


Figure 18:Interaction of Shape and Technique with 95% confidence error bars. (a) total trial completion time (b) total less control point reacquisition time We can also see from Table 4 that overall, even after control point reacquisition time removal, S2 still outperforms C1 by about 23%.
 
Mean Total Time
Standard Deviation
Standard Error
Technique
(seconds)
(seconds)
(seconds)
C1
11.72
5.09
0.23
S2
9.03
4.30
0.20

Table 4: Mean total trial completion times with reacquisition times removed for Experiment 2.

We also recorded the number of errors, i.e. the number of times that the participantsubject had to re-acquire control point(s) and to adjust the encompassing figure. Variance analysis showed that the only significant factor to error rate was technique (p <0.001). None of the other main effects or interaction terms was significant. On average participantsubjects made 4.2 adjustments with C1 technique and only 0.77 adjustments with the S2 technique in each trial of the task. Figure 19 and 20 illustrate the number of errors as a function experimental blocks and shape. The large error disparity between the two techniques was in conformity with our early analysis: With the one handed technique, due to the cognitive difficulty of mentally visualizing the control points forced upon the user, the participantsubjects often had to make a commitment to a control point that was not accurate so they had to re-acquire and adjust it later. In contrast, with the two handed technique, the need of searching for the control point became externalized. The user only had to rely on perceptual and motor skills to explore the accurate control points with their resulting consequence continuously displayed. Interestingly, even with the two handed technique, the participantsubjects still occasionally dropped the control points and then realized they were not on target.

Different from trial completion time, which was improved significantly over the 5 blocks of the experiment, the number of errors did not change significantly in the course of the experiment (p = 0.74, Figure 19), suggesting that the cognitive difficulty of projecting correct control points and the misperception of being on target were too inherent to overcome by learning in the experiment.

Figure 19: Means of Error in one trial with 95 confidence error bar as a function of experimental blocks
Figure 20: Means of Error with 95 confidence error bar as a function of target Figure shape Conclusions and Discussions

The two experiments presented in this paper provide rich information on how two-handed input differs from one-handed input. This section organizes our major conclusions around our initial hypotheses.

Hypothesis 1. Both two-handed techniques (S2 and T2) will outperform the conventional unimanual technique (C1). The results clearly support this hypothesis. As measured by total trial completion time, performance was about 17% faster in Experiment 1 and 39% faster in Experiment 2. This supports previous finding such as Buxton and Myers (1986) and Kabbash et al., (1994). It is important to note that the current finding is based on an entirely different class of tasks (area sweeping), thus establishing the research of two handed input on a broader ground. Hypothesis 2. Performance will improve with level of integration (chunking), from C1 to S2 and from S2 to T2. This hypothesis was only partially true. The hypothesis assumed that the higher the degree of chunking the interaction techniques supports, the more efficient the user can become. Our results showed such a prediction was not always correct. Certainly, performance improved when specifying the two corners of the swept out area were integrated (S2 – "Stretching with two hands" technique). However, no improvement was found when the sweeping sub-task was further integrated with the tool selection sub-task using the Toolglass technique. This suggests that the design of machine interface should support an appropriate the nature level of chunking. No additional advantage could be gained when two cognitively distinct subtasks (tool selection and sweeping) were integrated in the interface design.

Another factor that might reduce the effectiveness of the Toolglass technique is that even though an offset was built into the system, by offsetting the effective input location of the puck, the participantsubjects still expected the puck and stylus to physically overlap because on the screen the palette and the cursor overlap. This confusion could arise due to the fact the input space was shared by two hands, both stylus and puck operated within the same coordinate space. In fact, one participantsubject initially lifted the pen and attempted to click within the plastic cross-hairs area of the puck.

Hypothesis 3. Performance difference between the one-handed technique and the two two-handed techniques will become more pronounced as more mental visualization or planning is required, i.e. as cognitive difficulty of the task increases. This hypothesis was supported by the data. For the cognitively less demanding tasks (smaller size or shapes whose encompassing figure’s control points are easier to be mentally visualized) we see the performance of two-handed was similar or even inferior to that of one-handed. But as the tasks became more cognitively demanding (larger size, harder shape for mentally visualizing the correct control point placements), we see that two-handed has a significant performance gain (Figure 15 and Figure 16). Note that as size increases, in addition to increased cognitive loadvisualization load, the physical load of the task increases according to Fitts’ Law, in a logarithmic scale. But we see that the performance gap between one-handed and two-handed, as size increases, actually grows at a greater than logarithmic rate. In the second experiment, we only chose the target size and shape that were cognitively demanding. We also provided more repaid feedback to all conditions. As a result, we see an even greater difference between the one-handed and two-handed conditions. This again suggests that the performance difference between one-handed and two-handed techniques cannot be explained by physical efficiency alone.

In summary, the present work extends two-handed input into a new class of tasks. Furthermore, the experiments and analyses advance the theoretical understanding of two-handed input. As our literature review showed, two-handed input has made significant progress in the past decade. The introduction of the kinematic chain theory of human bimanual function greatly enriched the theoretical foundation of two-handed input design, in terms of the different functions each hand has. However, the KC theory does not explain why two hands can be better than one hand for computer input. The present work provides a complimentary perspective. We base our analysis of two-handed input on the concept of chunking and phrasing and argue that the advantage of two-handed input is both manual (physical) and cognitive. When a user can manipulate the entire object with two hands as an integrated chunk that is greater than the separate elements as in unimanual input, both manual (elimination of repeated control points reacquisition) and cognitive (reduction of mentally visualizing the control points) advantages were gained. Our experiments showed that as the task became more cognitively demanding (more visualization required), the advantage of two-handed input over one-handed input became more pronounced. We believe that to give users the ability to use both hands and manipulate the task at a natural level of chunking is a logical and an inevitable trend for a broad range of human-computer interaction applications.

Acknowledgement

We thank the anonymous reviewers and our editor Bonnie John for their helpful and constructive reviews which has substantially improved the paper.

References

Anderson, J.R. (Ed.) (1980). Cognitive Skills and their Acquisition, Hillsdale, N.J.: Lawrence Erlbaum & Associates.

Anderson, J.R. (1982). Acquisition of Cognitive Skill, Psychological Review, 89( 4), 369-406.

Bier, E.A., Stone, M.C., Pier, K., Buxton, W. and DeRose, T.D. (1993). Toolglass and magic lenses: The see-through interface. In Proceedings of SIGGRAPH ‘93, Annual Conference Series (Anaheim, California), Computer Graphics 27, 73-80.

Buxton, W. (1986). Chunking and phrasing and the design of human-computer dialogues. In Proceedings of the IFIP World Computer Congress (Dublin, Ireland), 475-480. Reprinted in Baecker, R., Grudin, J., Buxton, W. and Greenberg, S. (Eds.). (1995). Readings in Human-Computer Interaction: Towards the Year 2000. San Francisco, CA: Morgan Kaufmann, Inc., 494-499.

Buxton, W. and Myers, B.A. (1986). A study in two-handed input. In Proceedings of CHI ‘86 Conference on Human Factors in Computing Systems (Boston, Massachusetts), 321-326.

Card, S., Moran, T. & Newell, A. (1980). The Keystroke Level Model for User Performance Time with Interactive Systems, Communications of the ACM, 23(7), 396-410.

Chatty, S. (1994). Extending a graphical toolkit for two-handed interaction. In Proceedings of the ACM Symposium on User Interface Software and Technology (Marina del Ray, California), 195-204.

Dillon, R.F., Edey, J.D. and Tombaugh, J.W. (1990). Measuring the true cost of command selection: Techniques and results. In Proceedings of CHI ‘90 Conference on Human Factors in Computing Systems (Seattle, Washington), 19-25.

Fitts, P.M. (1954). The information capacity of the human motor system controlling the amplitude of movement. Journal of Experimental Psychology 47, 6, 381-391.

Fitts, P. M., & Seeger, C. M. (1953). S-R compatibility: spatial characteristics of stimulus and response codes. Journal of Experimental Psychology, 46, 199-210.

Flowers, K. (1975). Handedness and controlled movement. British Journal of Psychology, 66 (1), 39-52.

Guiard, Y. (1987). Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior 19, 4, 486-517.

Guiard, Y. and Ferrand, T. (1996in press). Asymmetry in bimanual skills. In Elliott, D. (Ed.) Manual Asymmetries in Motor Performance. Boca Raton, FL: CRC Press, 175-195.

Hinckley, K., Pausch, R., Goble, J.C. and Kassell, N.F. (1994). Passive real-world interface props for neurosurgical visualization. In Proceedings of CHI ‘94 Conference on Human Factors in Computing Systems (Boston, Massachusetts), 452-458.

Kabbash, P., Buxton, W. and Sellen, A. (1994). Two-handed input in a compound task. In Proceedings of CHI ‘94 Conference on Human Factors in Computing Systems (Boston, Massachusetts), 417-423.

Kirsh, D. and Maglio, P. (1994). On distinguishing epistemic from pragmatic action. Cognitive Science 18, 4, 513-549.

Krueger, M. (1983). Artificial Reality. Reading, MA: Addison-Wesley.

Owen, R., Kurtenbach, G., Fitzmaurice, G., Baudel, T. and Buxton, W. (1996). Bimanual manipulation in curve-editing tasks. Unpublished Manuscript.

Peters, M., Constraints in the performance of bimanual tasks and their expression in unskilled and skill subjects, The Quarterly Journal of Experimental Psychology (1985) 37A, 171-196.

Poulton, E.C. (1966). Unwanted asymmetrical transfer effects with balanced experimental designs. Psychological Bulletin 66, 1, 1-8.

Poulton, E.C. (1989). Bias in Quantifying Judgments. Hillsdale, NJ: Lawrence Erlbaum Associates.

Rosenbloom, P.S. (1985). The Chunking of Goal Hierarchies: a Model of Practice and Stimulus-Response Compatibility, Doctoral Dissertation, Dept. of Computer Science, Carnegie Mellon University.

Sachs, E., Roberts, A., and Stoops, D. 3-Draw: a tool for designing 3D shapes. IEEE Computer Graphics and Applications 11, 6 (Nov. 1991), 18-26.

Schmidt, R.A., Zelaznik, H., Hawkins, B. Frank, J.S. and Quinn, J.T. (1979). Motor-output variability: A theory for the accuracy of rapid motor acts. Psychological Review 86, 5, 415-451.

Shneiderman, B. (1983). Direct manipulation: A step beyond programming languages. IEEE Computer 16, 8, 57-69.

Simon, H. (1974). How big is a chunk? Science, 183, 482-488.

Smith, D.C., Irby, C., Kimball, R., Verplank, W. and Harslem, E. (1982). Designing the Star user interface. Byte 7, 4, 242–282.

Summers, J.J., T.M. Kennedy, T.M., Strategies in the production of a 5:3 polyrhythm, Human Movement Science 11 (1992) 101-112

Sutherland, I.E. (1963). Sketchpad: A man-machine graphical communication system. AFIPS Conference Proceedings, 23, 329-346.

Todor, J.I. and Doane, T. (1978). Handedness and hemispheric asymmetry in the control of movements. Journal of Motor Behavior 10, 295-300.

Wacom Corporation, Ltd. (1994). Wacom UD-1212R, UD-1218R/RE and UD-1825R Serial Tablet User’s Manual.