Haptics Commmunity Web Page Online Haptics Library

© 1995 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the Institute of Electrical and Electronics Engineers.

Abstract

Our group is interested in using haptic display for training tool use. Applications include training doctors to use tools during surgery, and training astronauts to use tools during EVA. This paper describes some of the challenges of creating realistic haptic perceptions of tool use. Many of these challenges stem from the importance of unilateral constraints during tool use. Unilateral constraints occur whenever rigid bodies collide, resisting the interpenetration of the bodies, but not holding the bodies together. To identify unilateral constraints, a tool/environment simulation must perform collision detection. To respond properly to a collision, the simulation must estimate the forces that ensue, and integrate the equations of motion. All of these computations must occur in real time, and the simulation as a whole must be stable (to ensure the user's safety). Approaches to these problems are described.

1. Introduction

A haptic display (or force reflecting interface) is a device which lets the user touch, feel and manipulate virtual environments. Many such devices have been developed in recent years, including but not limited to [1, 4, 10-13]. One promising area for the application of haptic display is tool use, in terms of both the design process and the training of new users. For example, designers can reduce prototyping time and costs by implementing new ideas in a virtual environment, rather than in a machine shop. Conventional VR can be and has been used in this way (see [17] for one example); however, for many tools, appearance alone doesn't allow a designer to understand how the tool will perform. In such cases, functionality is revealed by the physical interactions which the tool allows between the user and an environment. To explore this functionality, the ability to interact physically with virtual environments is indispensable.

As an illustration of this point, consider the recent use of VR methods to train Space Shuttle support personnel in procedures involving highly specialized hand tools. While some tools used in space are quite ordinary, others have unusual shapes and functions (e.g., various tools for emergency repairs). In the current VR training environment, tools are not represented at all, since this inclusion would require simulation of the interactions between virtual objects. For example, one merely points to a bolt that needs to be loosened, and it loosens itself. Clearly, this is useful for learning a complicated procedure, but not a physical skill. To develop a physical skill, haptic interaction is a necessary component of training.

In this paper, a number of the issues that arise in generating convincing haptic perceptions of tool use are discussed. The issues we choose to focus on are all related to one particularly salient feature of tool use: the ubiquity of unilateral constraints. Unilateral constraints occur whenever bodies collide, resisting the interpenetration of the bodies, but not holding the bodies together. We will show that unilateral constraints are closely related to the problem of system stability, the selection of a simulation technique, and the relatively unexplored problem of collision response.

These issues and others are reviewed in the next section. In section 3, a simple instance of unilateral constraint, the virtual wall, is considered. Analytical results regarding the stability of the virtual wall are extended to a broad class of virtual environments in section 4, and simulation techniques are discussed in section 5.

2. Issues in Tool Use

The purpose of this section is to point out a number of significant technical challenges associated with tool use. Interaction between a tool and its environment is characterized by relative motion, including collisions, sliding, penetration, cutting, etc. Real-time simulation of these various behaviors clearly poses a great challenge. To ensure a manageable discussion and a tractable scope, we will restrict attention to tools and environments in which all bodies are sufficiently rigid that changes in shape do not need to be considered. This continues to encompass a usefully broad class of tools.

The problem we are faced with, therefore, is the real-time simulation and haptic display of rigid body systems in which the bodies may interact via bilateral constraints (e.g., pin joints, slider joints) or unilateral constraints (e.g., surface-surface contact). A number of issues may be readily identified.

First, governing equations must not only be integrated in real-time, but must be formulated in real-time. This necessity develops because the act of making or breaking contact associated with a unilateral constraint changes the degrees of freedom of the system.

Second, in order to formulate equations in real-time, collisions must be detected in real-time. Fortunately, the computer graphics literature provides a wealth of sophisticated techniques for handling this problem.

Third, collision detection is not sufficient for equation formulation, because collisions can be detected (in real-time) only after a finite overlap of bodies occurs. It is necessary, therefore, to establish a method of collision response, which determines the appropriate kinematic constraint caused by the collision, and computes the appropriate reaction forces.

To elaborate on this point, suppose that bodies are represented by polygons. The difficulty of computing reaction forces relates to the number of vertices and edges involved in the collision. Consider the example shown below. It is relatively easy to determine that the appropriate response is to constrain the vertex to lie along the edge it penetrates.

Figure 2. Ambiguity in collision response.

Figure 3. More ambiguity in collision response.

A fourth issue is that the eigenvalues associated with constrained rigid body systems are either near zero (along directions of freedom) or infinite (along directions of constraint). The infinite eigenvalues can either be approximated by very large eigenvalues (this is tantamount to modeling constraints using stiff springs), or can be removed by finding an appropriate set of generalized coordinates.

Whichever simulation method is used, the haptic display hardware must be able to represent either complete freedom or complete constraint in any direction at any time, including rapid switches between the two, corresponding to making or breaking contact. Another way of saying this is that the mechanical impedance of the haptic display must be able to, under software control, exhibit a tremendous dynamic range: from near zero (complete freedom) to near infinite (complete constraint). We call the dynamic range of impedances which a particular device can implement its "Z-width."

Any haptic display will exhibit a finite Z-width. The real-time simulation acts like a feedback controller around the haptic display, and impedance limits are imposed by the stability of this controller. Thus, a haptic display cannot exhibit arbitrarily large or small impedances without a loss of stability, a condition which is generally intolerable from the user's perspective. Unfortunately, most devices built to date exhibit severely restricted Z-widths, making them completely unsuitable for tool simulation.

In the next two sections, the issue of Z-width is considered in some detail. Section 5 will return to the issues of simulation method and collision response.

3. Review of Virtual Wall Results

In the virtual wall problem, the "tool" is represented as a point restricted to motion along a line, and the "wall" is a unilateral constraint along that line. This is the simplest instance of unilateral constraint, and widely recognized as an important problem in the field of haptic display [7, 15, 16]. This section reviews our previous work on the virtual wall problem as a preliminary to the new results presented in section 4.

Our studies of the virtual wall have sought to answer the following question: how can a haptic interface be designed and a virtual wall implemented in software in order to maximize the ratio of in-contact impedance to out-of-contact impedance? Or, how can we build the stiffest possible virtual wall? To answer this, we must first understand what imposes limits on the achievable dynamic range. As mentioned above, limits are typically imposed by instability. Unfortunately, it is not possible to consider stability of the haptic display alone because system stability is very much affected by the human operator. A haptic display which is stable in isolation may become unstable when grasped by an operator. This is unfortunate because a human operator is difficult to model. Therefore, to make some headway on this problem, we have looked for conditions under which the haptic display, implementing a virtual wall, is passive [7,8].

Why would a virtual wall implementation not be passive? One very important reason is that a haptic display is a sampled-data system, rather than a continuous-time system. This means that there is an inherent delay between input and output, and that information is lost in sampling. In addition, the sensors used to measure the state of the display are imperfect. We have used the model shown in Figure 4 to study this problem. This model captures the sampled-data nature of haptic display, but not the sensor imperfections. We have used this model because it is the simplest model capable of representing the salient physical behavior. We have found that it lends vital insights into the issue of passivity.

Figure 4. Model of a one degree-of-freedom haptic display system implementing a virtual wall.

It is shown in [8] that a necessary and sufficient condition for the passivity of this model is:

(1)

We will consider a common implementation of a virtual wall composed of a virtual spring and damper in mechanical parallel, together with a unilateral constraint operator. A velocity estimate is obtained via backward difference differentiation of position, giving the following transfer function within the wall:

(2)

where K > 0 is a virtual stiffness, and B is a virtual damping coefficient (we will allow B to be positive or negative). Using this expression for H(z), the following result is found, after some manipulation:

(3)

One important conclusion that can be drawn from this analysis is that, to implement stiff, dissipative walls (high K, B), it is apparently helpful to maximize b and minimize T. Fast sampling is a standard objective, but maximizing damping goes against conventional wisdom. It is generally argued that the dynamics of a haptic interface should be dominated by the virtual environment (which is, after all, the programmed behavior we wish to display) rather than any inherent dynamics (which is considered parasitic). In other words, the interface hardware should be "transparent." Unfortunately, the notion of transparency places focus on mimicking the governing equations (e.g., state equations) of physical systems, but not on obeying underlying physical laws (such as conservation of energy). The addition of physical damping helps the sampled-data system to behave as physical law would dictate.

One rather glaring deficiency of this analysis, however, is that it has considered only high impedances, not low. While additional physical damping allows higher impedances to be implemented, it also increases the impedance of the haptic display. For the implementation shown in Figure 4, the minimum impedance is that of the display, unless negative gains are used. This, however, is precisely the solution: negative virtual damping may be used to compensate for the effect of physical damping in the region outside the wall. In fact, since K = 0, one may select B = -b, resulting in zero net damping (although this is borderline passive, and perfect cancellation is difficult to achieve in practice).

In summary, even the simplest version of a unilateral constraint demands careful attention to haptic display design as well as selection of simulation parameters. To achieve high impedances, it is important that the display incorporate physical dampers. To achieve low impe-dances, the effect of this damping must be compensated (this can be done with negative virtual damping, as described above, or by directly measuring the drag torque of the damper, and using this signal in a damping cancellation loop).

4. Robust Display of Complex Environments

Consider now the haptic display of a rigid tool interacting with a rigid environment (e.g., placing a wrench on a nut). This interaction is characterized by multiple unilateral constraints. The question arises: how can such a simulation be designed to ensure a suitable Z-width? One obvious approach is to model each unilateral constraint as a spring-damper, and select the stiffness and damping coefficients to be as large as possible without compromising passivity. Because the number of parameters is now quite large, and the system quite nonlinear, an analytical result is not feasible. Therefore, it will probably be necessary to use a trial-and-error approach to find appropriate values. This is precisely the manner in which most virtual environment simulations for haptic display are currently designed. Yet, even beyond its ad hoc and time-consuming nature, there are problems with this approach.

The most important problem is that it neglects the crucial role of geometry in determining apparent impedance. Consider the example shown in Figure 5, of a rigid peg placed in a rigid hole. Suppose that a shearing force is applied to the top of the peg. The apparent stiffness may be quite high when the peg is deeply seated, and quite low when the peg barely enters the hole, despite a consistent selection of unilateral constraint stiffness. A more sophisticated treatment of unilateral constraints is needed.

Figure 5. The stiffness felt at the tip of the peg depends on geometry, not just kinematic constraint.

The basic idea is illustrated in Figure 6. There are two key elements: one, the tool and environment are simulated by some method that is guaranteed to be discrete time passive, or nearly so; two, the handle of the virtual tool is connected to the handle of the haptic display via a multi-dimensional coupling consisting of stiffness and damping. The model of this coupling is strongly reminiscent of the virtual wall model.

Figure 6. Conceptualization of proposed haptic display and simulation structure

It is important to understand how this approach does, in fact, guarantee passivity of the sampled-data system. It does so because of the virtual coupling, which effectively limits the maximum impedance that need be exhibited by the haptic display. Thus, even when the impedance of the simulation is infinite, the impedance of the display need be only that of the virtual wall.

To understand this approach somewhat more deeply, and to place limits on the values of K and B, we will examine passivity in the case of a one degree-of-freedom system with linear environment dynamics. A model of this system is shown in Figure 7.

Figure 7. Model of a one degree-of-freedom haptic display implementing a tool simulation consisting of a "virtual coupling" (H(z)) and a discrete time passive simulation (E(z)).

(4)

It is also assumed that for , which is equivalent to requiring that the simulation be discrete time passive. A final assumption is that the state of the virtual coupling and the state of the virtual environment are updated in synchrony. This is important because it will, in general, require the solution of implicit algebraic equations.

The tool that we will use to find passivity conditions is the "structured singular value" ( ) [9]. This analysis begins by replacing the operator with a "pseudo-operator" which is a linear time-invariant system known only to be passive. It turns out that the conditions for the stability of the operator-display-simulation system that results are the same as the passivity conditions for the display-simulation system [5].

The second step is to transform the coordinates used to describe the input/output behavior at both port 1 and port 2 of the virtual coupling. The objective of this transformation is to replace the set of possible environment behaviors (i.e., set of possible E(z)'s) with the set of stable linear, shift-invariant operators having gain less than one (i.e., the set of transfer functions , constrained only to be stable and satisfy ); and likewise, to replace the set of possible operator/display behaviors with the same set . The motivation for doing this is that is much easier to describe and work with than either the set of environment behaviors or the set of operator/display behaviors. Details of both these transformation are given in [6].

Having performed the appropriate transformations, the block diagram in Figure 7 can be manipulated into the form shown in Figure 8.

Figure 8. Block diagram equivalent to that in Figure 7, showing the applicability of structured singular value analysis.

(5)

Details of computing the structured singular value are given in a number of places, perhaps most succinctly in the manual for the Matlab's -Analysis and Synthesis Toolbox [2].

While an analytical result to equation 5 has not yet been found, extensive numerical analysis has shown that it is satisfied under precisely the same conditions as those derived for the passivity of the virtual wall (equation 3). This is a convenient result, because it not only extends the analytical results for the virtual wall to a far broader class of systems, but suggests that the efforts to design and implement haptic display hardware for the display of stiff virtual walls have been useful in a much more general sense.

5. Discussion and Conclusions

There is at least one significant difficulty with the result of the previous section. This is the need for backwards difference integration techniques, as was mentioned. Backwards difference techniques (commonly used to integrate stiff systems of differential equations) lead to sets of implicit algebraic equations whose solution can be time-consuming. Thus, the questions arise: do efficient backwards difference techniques exist? How much more difficult is it to ensure stability if a forward-difference (explicit) method of integration is used? These questions merit considerably more attention than can given here, but a few brief comments are in order.

Regarding the second question, we have implemented two classes of rigid body simulations, both using explicit methods. The first class is a "penalty method" [3], in which constraints are modeled as stiff springs and dampers. Using penalty methods and explicit integration, it is essentially impossible to guarantee stability. We are currently investigating an approach to penalty methods which uses implicit integration. The second class is "coordinate partitioning" [18], in which generalized coordinates are found at every time step. Although also explicit, this method generates near-zero eigenvalues and proves to be much more robust in practice. Unfortunately, the computational overhead is large, leading to slow update rates and a rather soft virtual coupling.

Mitrich and Canny [14] have recently proposed an approach to simulation which may address all of the above weaknesses. In their method, constraints are not implemented at all; rather, when collisions occur, impulses are computed sufficient to prevent interpenetration. It is, however, relatively easy to ensure that impulses do not add energy to a rigid body system. Also with this approach, the state of each rigid body can be integrated individually, making implicit methods a reasonable choice. Thus, it should be possible to guarantee discrete time passivity while achieving relatively high update rates. The implementation of an impulse-based simulation and interface to a haptic display is a topic of current research in our laboratory.

Acknowledgments

This work has been supported by the National Science Foundation (CMS-9403730) and by the National Aeronautics and Space Administration (NAG 9-734).

References

1. Adelstein, B. D. and M. J. Rosen. Design and Implementation of a Force Reflecting Manipulandum for Manual Control Research. ASME Winter Annual Meeting. Anaheim, California (1-12) (1992)

2. Balas, G. J., J. C. Doyle, K. Glover, A. Packard and R. Smith. u-Analysis and Synthesis Toolbox. The MathWorks, Inc. Natick, MA (1991)

3. Barzel, R. and A. Barr. A Modeling System Based on Dynamic Constraints. Computer Graphics 22(4):179-187 (1988)

4. Brooks, F. and e. al. Haptic Displays for Scientific Visualization. 24(4):177-185 (1990)

5. Colgate, J. E. Robust Impedance Shaping Telemanipulation. IEEE Trans. on Robotics and Automation 9(4):374-384 (1993)

6. Colgate, J. E. Coordinate Transformations and Logical Operations for Minimizing Conservativeness in Coupled Stability Criteria. Journal of Dynamic Systems, Measurement and Control 116(4):63-649 (1994)

7. Colgate, J. E. and J. M. Brown. Factors Affecting the Z-width of a Haptic Display. International Conference on Robotics and Automation. San Diego, CA (3205-10) IEEE R&A Society (1994)

8. Colgate, J. E. and G. G. Schenkel. Passivity of a Class of Sampled-Data Systems: Application to Haptic Interfaces. American Control Conference. Baltimore (1994)

9. Doyle, J. C. Structured Uncertainty in Control System Design. 24th Conference on Decision and Control. Ft. Lauderdale, Florida (260-265) (1985)

10. Ellis, R. E., O. M. Ismaeil and M. G. Lipsett. Design and Evaluation of a High-Performance Prototype Force-Feedback Motion Controller. Advances in Robotics, Mechatronics and Haptic Interfaces, 1993. Kazerooni, Colgate and Adelstein ed. ASME. (1993)

11. Kelley, A. J. and S. E. Salcudean. On the Development of a Force-Feedback Mouse and Its Integration Into a Graphical User Interface. International Mechanical Engineering Congress and Exposition. Chicago (287-294) ASME (1994)

12. Millman, P. A., M. C. Stanley, P. E. Grafing and J. E. Colgate. A System for the Implementation and Kinesthetic Display of Virtual Environments. SPIE Conference 1833: Telemanipulator Technology. Boston (49-57) (1992)

13. Minsky, M., M. Ouh-young, O. Steele, J. F.P. Brooks and M. Behensky. Feeling and Seeing: Issues in Force Display. Computer Graphics 24(2):235-243 (1990)

14. Mirtich, B. and J. Canny. Impulse-based Dynamic Simulation. (1994). University of California, Berkeley, UCB:CSD-94-815

15. Rosenberg, L. B. and B. D. Adelstein. Perceptual Decomposition of Virtual Haptic Surfaces. IEEE Symposium on Research Frontiers in Virtual Reality. San Jose, CA (1993)

16. Salcudean, S. E. and T. D. Vlaar. On the Emulation of Stiff Walls and Static Friction with a Magnetically Levitated Input/Output Device. International Mechanical Engineering Exposition and Congress. Chicago, IL (303-310) ASME (1994)

17. Tanner, S. The Use of Virtual Reality at Boeing's Huntsville Laboratories. IEEE Virtual Reality Annual International Symposium. Seattle, Washington (14-19) (1993)

18. Wehage, R. and E. J. Haug. Generalized Coordinate Partitioning for Dimension Reduction in Analysis of Constrained Dynamic Systems. Journal of Mechanical Design 104:247-255 (1982)