The Influence of Muscle Groups on Performance of Multiple Degree-of-Freedom Input
Shumin Zhai Paul Milgram William Buxton Department of Industrial Engineering and Department of Computer Science Department of Industrial Engineering Alias|Wavefront Inc. University of Toronto University of Toronto, 210 King St. East Toronto, Toronto Toronto Ontario Ontario Ontario Canada Canada Canada MS4 1A4 MS4 1A4 M5A 1J7 ABSTRACT
The literature has long suggested that the design of computer input devices should make use of the fine, smaller muscle groups and joints in the fingers, since they are richly represented in the human motor and sensory cortex and they have higher information processing bandwidth than other body parts. This hypothesis, however, has not been conclusively verified with empirical research. The present work studied such a hypothesis in the context of designing 6 degree-of-freedom (DOF) input devices. The work attempts to address both a practical need - designing efficient 6 DOF input devices - and the theoretical issue of muscle group differences in input control. Two alternative 6 DOF input devices, one including and the other excluding the fingers from the 6 DOF manipulation, were designed and tested in a 3D object docking experiment. Users' task completion times were significantly shorter with the device that utilised the fingers. The results of this study strongly suggest that the shape and size of future input device designs should constitute affordances that invite finger participation in input control.Keywords
Input devices, 3-D interface, 6 DOF input, motor control, muscle group differences, hand, fingers, arm, homunculus model.
Increasingly, the user interfaces that we are designing and using involve higher degrees of freedom than were found in first generation GUIs. This places a higher load on the operators of such systems. Consequently, it is all the more important that the motor and cognitive resources of the operator be used to their greatest effect. Many studies have already been conducted which address issues in multiple DOF input (e.g. [11], [12], [14], [25], [26] and [27]).
The study reported in this paper adds to this literature. It investigates how human performance in 6 DOF tasks varies according to the muscle groups employed. The implications of this research are very practical. If performance for a given task is higher when a particular muscle group is employed, then future input devices can be tailored accordingly, with affordances [8] that encourage the use of that group.
The present study was inspired by Card, Mackinlay and Robertson's "morphological analysis" of the design space of input devices, which suggested "a promising direction for developing a device to beat the mouse by using the bandwidth of the fingers" [2]. This prediction was based upon the well-established "homunculus model" from neurophysiology (Figure 1) and some motor bandwidth studies using Fitts' law tasks. However, empirical HCI studies that support such a prediction have not been conclusive. One candidate device which affords finger based manipulation and could therefore conceivably "beat" the mouse is a pen shaped 2 DOF input device such as the stylus studied by MacKenzie, Sellen and Buxton [19]. In their comparison of a stylus with a mouse, however, the results were inconclusive: the stylus outperformed the mouse in dragging but not in pointing tasks. Furthermore, in the context of conventional GUIs, even if the stylus were shown to result in higher performance, the issue may well be moot. This is because the time to acquire the device (i.e. the time for moving between the keyboard and pointing device, which is much longer with the stylus) may dominate the overall performance. We feel that such "side effects" and the collective user experience with the mouse as the "standard" 2 DOF computer input device may make the replacement of the mouse with a finger operated device a futile pursuit.
Figure 1 Homunculus model of somatosensory (left) and motor (right) cortex (Adapted from [23], G. H . Sage, Introduction to Motor Behavior, ©1977 Addison-Wesley Publishing Company Inc. Reprinted by permission of Addison-Wesley Publishing Company, Inc.)
In the present study, for several reasons, we have chosen to focus
on 6 DOF tasks. First of all, there is not yet an accepted standard 6 DOF
input device. Secondly, few would disagree that 6 DOF input tasks are much
more demanding than 2 DOF input tasks. Consequently, if one particular
design factor results in a slight advantage in a 2 DOF input task, that
same subtle advantage may manifest itself to a much larger extent in 6
DOF input tasks.
One of the first studies on the effects of different body parts in manual control was done by Gibbs [6]. In a one dimensional target acquisition task, Gibbs studied task performance of three different body parts: the thumb (activated by the carpometacarpal joint), the hand (activated by the wrist), and the forearm (activated by the elbow), in both position and rate control systems with various control gains and time delays. The performance ranking in Gibbs’ study was, in decreasing order: hand, forearm, thumb.
Hammerton and Tickner later replicated Gibbs' study in a 2 DOF target acquisition task [9]. Although Gibbs and Hammerton et al subsequently argued about their experimental methodology [7][10], the two studies in fact arrived at very similar conclusions, that performance with the hand (wrist movement) was superior to that of the thumb and the forearm. This advantage was greater for more difficult tasks, such as those with long time delays [9]. Note that both studies found that the wrist was more effective than the thumb. Neither Gibbs nor Hammerton and Ticker included fingers in their studies, however.
The motor performance of different limbs has also been investigated in various Fitts' law studies. Fitts' law established the simple linear relationship: MT = a + b ID in repetitive tapping tasks, where MT is the movement time, ID = log2(2A/W) is the Index of Difficulty, A is the movement amplitude and W is the width of the target area [4]. The slope parameter b, in units of seconds/bit, is the inverse of the motor system information processing rate. Fitts' law studies have typically found this rate (1/b) to be in the vicinity of 10 bits/second when the arm is involved in the movement.
Fitts speculated that other limbs, such as fingers, may show different processing rates [4]. Subsequent studies in fact supported this hypothesis. Langolf, Chaffin, and Foulke investigated the Fitts' law relationship using amplitudes of A = 0.25 cm, A = 1.27 cm and A > 5.08 cm [16]. For the first two amplitudes, the experiment was carried out using a microscope. For the large amplitude (>5.08 cm), the experiment was carried with direct vision. Langolf and colleagues observed that for A = 0.25 cm subjects moved the stylus tip (a 1.1 mm peg) primarily with finger flexion and extension. For A = 1.27 cm, flexion and extension of both wrist and fingers occurred. For A > 5.08 cm, the forearm and upper arm were involved in the movements. On the basis of this method of allocating actuation to different muscle groups by controlling the range of movement, Langolf and colleagues concluded that the information processing rates (1/b) for the fingers, wrist, and arm were 38 bits/sec, 23 bits/sec and 10 bits/sec respectively (see Figure 6 in [16]). This study has been widely cited in the literature (e.g. [1]; [15]; [2]) as evidence that fingers are among the most dextrous organs. Card and colleagues recently reviewed Fitts' law studies with various body parts (finger, wrist, arm, neck) and pointed out the limitations of the widely used computer input device - the mouse [2].
In summary, both neurophysiological studies (the homunculus model) and Fitts’ law studies suggest that use of the small muscle groups (fingers and thumbs) should result in better performance than the large muscle groups (arm and shoulder). However some studies in manual control, such as Gibbs' study [6] and Hammerton and Tickner's study [9], are not completely consistent with such a prediction.
Due to the theoretical motivation, most studies in the literature tend to compare performance of different muscle groups against each other. From a practical and ecological point of view, such a contrast is not necessary for the design of 6 DOF input devices. The human upper limb as a whole (from shoulder to finger tips) has evolved to be highly dextrous yet powerful. Every part of it has its purpose and function. What is needed in input device design is to make use of all parts of the associated limb, according to their respective advantages. The larger muscle groups that operate the wrist, elbow, and shoulder have more power and a larger range of movement. The smaller muscle groups that operate the fingers and thumb have more dexterity. When all the parts work in synergy, movement range and dexterity can both be maximised.
Figure 2 The 6 DOF glove
The Glove design resembles many of the common virtual reality input devices. When using the Glove, all translation and rotation operations are carried out by the user's shoulder, elbow and wrist, i.e., the gross joints and muscle groups in the human limb. Other than pressing the T-button, the smaller, finer joints and muscle groups on the fingers were not utilised for the 6 DOF manipulation.
The T-button was an essential component of the input glove. Since the Glove requires rotation to be made with the wrist, the elbow and the shoulder, its range of rotation is limited. Whenever a limit is reached, the user needs the clutch to disengage the manipulated object (by releasing the T-button under the fingers) and restore the hand to a more comfortable posture, and then recommence the manipulation (by re-closing the fingers). This is very similar to lifting a 2 DOF mouse and starting from a new position on the mouse pad. We refer to this process as “re-clutching”.
The majority of existing designs of freely moving 6 DOF devices, such as the “Bat” [25], the “Cricket” [3] and the 3D mouse [17] are similar to the glove design in assigning wrist, elbow and shoulder muscle groups for manipulating the six degrees of freedom; however, none of these make use of the fingers.
Figure 3 The FingerBall
The ball shape was chosen because a symmetrical ball shape can easily be grasped and manipulated by the fingers in all directions. The FingerBall is designed to be held and moved (rolled) by the fingers, wrist, elbow and shoulder, in postures that have been classified as "precision grasp", as opposed to "power grasp" [18]. Precision grasping, while holding objects with the finger tips, places emphasis on dexterity and sensitivity. In contrast, power grasping, while holding objects against the palm, puts emphasis on security and power. The FingerBall has been designed with a versatile shape that is compatible with a variety of virtual object shapes. This approach of choosing a versatile shape is an alternative to the approach of making "props" that are designed to resemble features of virtual objects being manipulated [11]. The FingerBall instead is similar to the concept of "bricks" [5], which are universal physical handles to various virtual objects.
To take maximum advantage of finger operations, two additional features would be desirable. One is to make the ball tetherless, so that the user can roll it freely between her fingers without interference. The second desirable feature is that the ball be made of an elastic, conductive material, so that the entire ball functions as a button that can be squeezed from any direction. Since enabling technology for wireless design is not easily available, the FingerBall currently uses the Ascension Bird™ tracker mounted in the centre of a sponge filled ball 6 cm in diameter. The entire FingerBall weighs 66g. The cord of the Bird is pointed away from the hand in the null position, so as to maximise the range of rotation without interference from the cord (Figure 3). Furthermore, since the FingerBall can be rotated up to 180 degrees in any direction, it did not require a clutch as in the glove design for the task used in the following experiment. It was therefore not necessary to implement a button for the FingerBall in the experiment.
The 3BallTM, manufactured by Polhemus [20], is a commercial product similar to the FingerBall design. The limitation of the 3Ball is its fixed button location. In order to access this button, users can not freely roll the ball between their fingers. Another spherical implementation of a 6 DOF tracker, the "Cue Ball," was demonstrated by Dan Venolia at the CHI'90 Interactive Experience.
Figure 4 Experiment Set-up
Since the overall objective of this experiment was to evaluate 6 DOF input interfaces, the emphasis in designing the display was to provide the largest possible number of 3D spatial cues. This was to ensure that the task performance was driven predominantly by differences in input controller conditions rather than by difficulties in perceiving depth information. A 120 Hz sequential switching stereoscopic display was employed, which has been shown to be a necessary feature for this kind of experiment. To enhance the 3D effect, perspective projection and interposition cues were also implemented. The tetrahedra were drawn in wireframe so that all edges and corners of the objects could be perceived simultaneously. Subjects were asked to sit on a chair approximately 60 cm away from the computer screen for all experimental conditions (Figure 4).
Figure 5 The Experimental Task
At the beginning of each experimental trial the cursor appeared in the centre of the 3D space while the target randomly appeared in one of five pre-set locations (14 cm, 11.5 cm, 12 cm, 7 cm, and 10 cm away from the centre of the cursor in five arbitrary directions) and orientations (110, 98, 106, 140, and 115 degrees mismatched with the cursor about five arbitrary axes). In contrast to the data gathering trials, during practice sessions the target appeared in completely random locations and orientations.
During the trial, whenever a corner of the cursor entered into the tolerance volume surrounding the corresponding corner of the target, the star on that corner changed its colour as an indication of capture. Whenever all four corresponding corners stayed concurrently matched for 0.7 seconds, the trial was deemed completed. At the end of each trial, the trial completion time was printed on the screen. The beginning of each trial was signalled with a long auditory beep and the end of each trial was signalled with a short beep.
Figure 6 shows the subjects' mean trial completion times for each of the five tests. On average, task completion times were clearly shorter for the FingerBall than for the glove in each of the five tests. Repeated measure variance analysis showed that overall performance scores for the two devices were significantly different: F(1, 8) = 26.554, p < 0.005.
Figure 6 Task completion times with FingerBall and Glove (data from all 10 subjects)
With both modes, subjects significantly improved their performances over the course of the five experimental phases: F(4, 32) = 34.04, p < 0.0001. In addition, the performance differences between the two modes were independent of experimental phase, as indicated by the absence of a significant Device x Phase interaction: F(4, 32) < 1, p > 0.5. The performance differences between the two modes were also independent of initial target location / orientation: F(4, 32) = 1.28, p = 0.3.
Other significant factors included Block: F(1,8) = 26.44, p < 0.001. As stated earlier, each test consisted of two blocks of trials. Completion times in the second block were significantly shorter than for the first block, due to an obvious learning effect.
The presentation order of the two modes was not significant: F(1,8) = 2.2, p = 0.17, but the Order x Device interaction was significant: F(1,8) = 22.587, p < 0.005. This could imply the presence of an asymmetricalskill transfer as an artifact of the within-subjects design, a factor which is an often overlooked and which can result in misleading conclusions in such experimental research. As Poulton [21, 22] argued, with a within-subjects design, the actual skill transfer from one condition to another might not be symmetrical, even though subjects’ exposures to the two conditions are ostensibly equalised.
In the present experiment, the Order x Device x Phase interaction is also significant (F(4,32) = 8.7, p < 0.0001) indicating that, if there was an asymmetrical transfer effect, it varied with experimental phase. It is thus very likely that asymmetrical transfer might have been a significant factor in the early experimental phases but not in the later phases. Two approaches were therefore taken, respectively described in the following two subsections, to test if asymmetrical skill transfer was likely to have been the only cause of the performance difference between the FingerBall and the Glove.
Figure 7 shows the results after discarding half of the data in this fashion. Results of a repeated measure variance analysis based on these between-subjects data were consistent with the early analysis, i.e., completion times with the FingerBall were significantly shorter than the glove: F(1,8) = 3.6, p < 0.05. Learning apparently did not reduce this difference, as the Phase x Device interaction was not significant: F(4, 32) < 1, p > 0.5.
Figure 7 The FingerBall vs. the glove with 5 subjects in each group (between-subjects)
On the basis of all three of the above analyses, we can therefore confidently conclude that the FingerBall significantly outperformed the Glove in the experiment.
This issue had in fact been considered during the design stage of the experiment, however, and the re-clutching times (from the moment of disengaging to the moment of re-engaging) accumulated in each trial were measured and recorded during each trial. In the following analysis, the re-clutching times were subtracted from the trial completion times for the Glove condition. The resulting net score is labelled as "C-R time". For the FingerBall condition, for which no clutching was necessary, C-R time was identical to the original completion time itself. Figure 8 shows the mean completion time with the FingerBall, the mean completion time with the Glove, and the re-clutching time with the Glove, all from Test 5. The mean re-clutching time in Test 5 was 1.06 seconds, accounting for 10.7% of the mean completion time with the Glove (Figure 8).
Figure 8 Mean completion time in Test 5
Note that the C-R time measure is biased against the FingerBall condition, for two reasons. First, with the FingerBall, the re-clutching process still exists effectively, although not explicitly. From time to time subjects had to move their fingers to different parts of the ball's surface to make further rotation. This effort (and associated time) was not taken into account by the adjusted C-R time measure, since no explicit re-clutching time could be measured. Second, during the re-clutching time with the Glove, subjects were not necessarily idle but probably were instead engaged in mentally making decisions about what to do next. Since it is known that mental rotation takes up a certain amount of time [24], this time may overlap with the re-clutching time in the Glove condition and is therefore reduced in the C-R time measure. Nevertheless, C-R time serves as a conservative measure to test if the use of fingers really was advantageous. If the FingerBall still outperformed the Glove, as measured by C-R time, the advantage of using fine joints must therefore exist. The converse may not be true, however.
Figure 9 shows the performance differences between the FingerBall and the Glove as measured by C-R time. As can be seen, the mean completion times with the FingerBall were still shorter than the mean C-R times with the Glove. Repeated measure variance analysis of C-R times collected in Test 5 (final phase of the experiment) showed that the difference between completion times with the FingerBall and the C-R times with the Glove was still significant: F(1,8) = 5.324, p < 0.05. Neither the order of presentation nor its interaction with the device was statistically significant, suggesting that this difference was not caused by asymmetrical skill transfer. This analysis therefore further supports the conclusion that the use of different muscle groups was indeed a major cause of the superior performance of the FingerBall as compared to the Glove.
Figure 9 Comparison between the FingerBall and the Glove after discounting re-clutching time with the Glove
Figure 10 Subjective Ratings of FingerBall vs. Glove
Upward arrows indicate that the FingerBall was preferred
2. Card, S. K., Mackinlay, J. D., and Robertson, G. G. A morphological analysis of the design space of input devices. ACM Transactions on Information Systems, 9, 2 (1991), 99-122.
3. Digital Image Design Inc. The Cricket, product information. New York, NY., 1993.
4. Fitts, P. M. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, (1954), 381-391.
5. Fitzmaurice, G. W., Ishii, H., and Buxton, W. Bricks: Laying the foundation for graspable user interfaces, in Proc. of CHI'95: ACM Conference on Human Factors in Computing Systems (Denver, Colorado, USA, 1995), pp. 442-449.
6. Gibbs, C. B. Controller design: Interactions of controlling limbs, time-lags and gains in positional and velocity systems. Ergonomics, 5, 2 (1962), 385-402.
7. Gibbs, C. B. Comments on ‘An investigation into the comparative suitability of forearm, hand and thumb controls in acquisition tasks’ by Hammerton and Tickner. Ergonomics, 10, 4 (1967), 431-432.
8. Gibson, J. J. The ecological approach to visual perception. Houghton Mifflin Company, Boston, 1979.
9. Hammerton, M. and Tickner, A. H. An investigation into the comparative suitability of forearm, hand and thumb controls in acquisition tasks. Ergonomics, 9, 2 (1966), 125-130.
10. Hammerton, M. and Tickner, A. H. Reply to Comments by C.B. Gibbs. Ergonomics, 10, 4 (1967), 433.
11. Hinckley, K., Pausch, R., Goble, J. C., and Kassell, N. F. Passive real-world interface props for neurosurgical visualization, in Proc. of CHI'94: ACM conference on Human Factors in Computing Systems (Boston, 1994), pp. 452-458.
12. Hinckley, K., Pausch, R., Goble, J. C., and Kassell, N. F. A survey of design issues in spatial input, in Proc. of ACM Conference on User Interface Software and Technology (1994), pp. 213-222.
13. Howell, D. C. Statistical Methods for Psychology (Third ed.). PWS-KENT Publishing Company, Boston, 1992.
14. Jacob, R. J. K., Sibert, L. E., McFarlane, D. C., and Mullen, M. P. Integrality and separability of input devices. ACM Transactions on Computer-Human Interaction, 1, 1 (1994), 3-26.
15. Keele, S. W. Motor Control, in K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of Perception and Human Performance. (pp. 30.1-30.60). John Wiley & Sons, New York, 1986.
16. Langolf, G. D., Chaffin, D. B., and Foulke, J. A. An investigation of Fitts' law using a wide range of movement amplitudes. Journal of Motor Behavior, 8, 2 (1976), 113-128.
17. Logitech. 2D/6D Mouse technical reference manual. In Fremont, CA 94555: Logitech Inc., 1991.
18. MacKenzie, C. L. and Iberall, T. The Grasping Hand. North-Holand, Amsterdam, 1994.
19. MacKenzie, I. S., Sellen, A., and Buxton, W. A comparison of input devices in elemental pointing and dragging tasks, in Proc. of CHI’91: ACM Conference on Human Factors in Computing Systems (New Orleans, Lousiana, 1991), pp. 161-166.
20. Polhemus. 3BallTM, product information. Polhemus, Cochester, VT., 1993.
21. Poulton, E. C. Unwanted asymmetrical transfer effects with balanced experimental designs. Psychological Bulletin, 66, 1 (1966), 1-8.
22. Poulton, E. C. Bias in Quantifying Judgements. Lawrence Erlbaum Associates, Hove and London, 1989.
23. Sage, G. H. Introduction to Motor Behavior, A Neuropsychological Approach (2nd ed). Addison-Wesley Publishing Company, 1977.
24. Shepard, R. N. and Metzler, J. Mental rotation of three-dimensional objects. Science, 171, 3972 (1971), 701-703.
25. Ware, C. Using hand position for virtual object placement. The Visual Computer, 6, (1990), 245-253.
26. Ware, C. and Jessom, D. R. Using the bat: a six dimensional mouse for object placement. IEEE Computer Graphics and Applications, 8, 6 (1988), 155-160.
27. Zhai, S. Human Performance in Six Degree of Freedom Input Control. Ph.D. Thesis , University of Toronto,1995.
28. Zhai, S. and Milgram, P. Human performance evaluation of manipulation schemes in virtual environments, in Proc. of VRAIS’93: the first IEEE Virtual Reality Annual International Symposium (Seattle, USA, 1993), pp. 155-161.
29. Zhai, S. and Milgram, P. Input Techniques for HCI in 3D Environments, in Proc. of Conference Companion of CHI'94: ACM conference on Human Factors in Computing Systems (Boston, 1994), pp. 85-86.