Active Voodoo Dolls: A Vision Based Input Device For Non-Rigid Control

For human-computer interaction, a wide variety of input devices have been invented. One category of these devices are those which transform intentional human motion into a measurable analog quantity. Not only must such a device be accurate, but also intuitive in such a way that the motion of the device corresponds to the motion which the human is controlling. Basic analog joysticks and mice have two degrees of freedom, which correspond to x and y axis of the screen. Spaceorbs have six degrees of freedom, three translational and three rotational, which are determined by the position and orientation of a spring mounted rubber ball with internal sensors. More complex and expensive devices such as haptic feedback pens and polhemus devices can also provide more than six degrees of freedom. All of these devices have the limitation that they only sense rigid, or in the best case, articulated motion.

Although it is theoretically possible to build a device to measure more degrees of freedom in motion (bending, twisting, etc.), vision based techniques offer an inexpensive alternative to control without the wiring and cumbersome devices.

Active Blobs can be used as an input device for non-rigid control that can be used for animation and video game applications. The user grasps a soft, squishable object in front of a camera which can be moved and deformed in order to specify motion. Active Blobs, a non-rigid tracking technique is used to recover the position, rotation and deformations of the object. The resulting transformations can be applied to a texture mapped triangular mesh, thus allowing the user to control it interactively. Our use of texture mapping hardware in tracking allows us to make the system responsive enough for interactive animation and video game character control.

Readers are referred to the technical report. for a detailed description of the approach.

Figure 1: Animating a bunny character via direct manipulation of a spongy object. The system observes the user manipulation of a deformable spongy object via a camera; representative frames from such a video sequence are shown in (a). The system then tracks the motion of the object by tracking a deformable region as shown in (b). The recovered deformation parameters are then applied to the graphics model for animation as shown in (c).

Figure 2: Animating a three-dimensional animal cracker box via direct manipulation of a spongy object. The video sequence is shown in row (a). The deformable region used for tracking is in row (b). In row (c) the deformation parameters are applied to two dimensions of a three-dimensional animal cracker box.

[ Active Blobs Home | Contact | IVC Home ]

Page Created by: John Isidoro
Last Modified: Mar 13, 1998