Active Blobs:
Approach

 IVC Logo
Active Blobs employ a new region-based approach to nonrigid motion tracking. Shape is defined in terms of a deformable triangular mesh that captures object shape plus a color texture map that captures object appearance. Photometric variations are also modeled. Nonrigid shape registration and motion tracking are achieved by posing the problem as an energy-based, robust minimization procedure. The approach provides robustness to occlusions, wrinkles, shadows, and specular highlights. The formulation is tailored to take advantage of texture mapping hardware available in many workstations, PC's, and game consoles. This enables nonrigid tracking at speeds approaching video rate.

The following is a brief overview of the active blobs approach. Readers are referred to the technical report. for a detailed description of the approach.

input image triangle mesh texture mapped model
Figure 1: Model construction using a color image. From left to right: a.) input image with region of interest overlaid, b.) resulting triangle mesh, c.) texture mapped model.

The construction of an example active blob model is shown above. The input to blob construction is an example image plus segmentation information --- provided as a binary support region or as a contour that encloses the shape. The input can also include interior feature points to be used as nodes in the triangular mesh. In this example, the user circled the object of interest.

A 2D active blob model is then constructed using a modified Delaunay triangular meshing algorithm. To deform the model, we deform this mesh. Nonrigid deformation of the mesh can be specified in terms of parametric functions; e.g., affine deformations, eight parameter projective deformations, application-specific deformations [Black95], finite element modal deformations [Pentland91,Sclaroff95], or principal deformations derived from a statistical analysis over a training set of shapes [Cootes92,Martin94,Nastar95].

The blob's appearance is then captured as a color texture map and applied directly to the triangulated model. A blob warp is defined as a deformation of the mesh and then a bilinear resampling of the texture mapped triangles. By defining image warping in this way, it is possible to harness hardware accelerated triangle texture mapping capabilities becoming prevalent in mid-end workstations, PC's, and computer game consoles ({e.g.}, Nintendo 64).

Tracking is then posed as the problem of active blob registration. The registration procedure minimizes a function that accounts for both the priors on shape (the deformation parameters) and the priors on appearance (the color texture map). Through the use of a robust error norm, registration can be made robust to specular highlights, shadows, some photometric variations, and small occlusions. Furthermore, the use of color imagery enables tracking in situations where grayscale tracking might be less robust. For details of the active blobs tracking formulation, readers are referred to the technical report.

input input input input
track track track track
Figure 2: Nonrigid tracking with an active blob. This figure shows every fifteenth frame in a tracking sequence. For visualization purposes, an outline of the active blob is shown overlaid on the input images in the top row. The resulting active blob tracking is shown below each input image.

An example of nonrigid tracking with an active blob is shown in Figure 2 above. The user defined a rectangular region of interest. A finite element modal parameterization was then employed for tracking. As can be seen, the blob model tracks the bag of candy quite well, despite nonrigid deformation, wrinkles, shadows, and specular highlights.

Click here to see more tracking examples.

References

  1. T. Cootes, D. Cooper, C. Taylor, and J. Graham. "Trainable method of parametric shape description." In Image and Vision Comp., 10(5):289--294, 1992.

  2. J. Martin, A. Pentland, S. Sclaroff, and R. Kikinis. "Characterization of Neuropathological Shape Deformations," In IEEE Trans. Pattern Analysis and Machine Intelligence, (to appear).

  3. M. Black and Y. Yacoob. "Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion." In Proc. ICCV 95.

  4. C. Nastar and A. Pentland. "Matching and recognition using deofmrable intensity surfaces." In Proc. IEEE Symposium on Computer Vision, 1995.

  5. A. Pentland and S. Sclaroff. "Closed-form solutions for physically-based shape modeling and recognition." In IEEE Trans. Pattern Analysis and Machine Intelligence, 13(7):715--729, 1991.

  6. S. Sclaroff and A. Pentland. "Modal Matching for Correspondence and Recognition." In IEEE Trans. Pattern Analysis and Machine Intelligence, 17(6):545--561, 1995.

  7. S. Sclaroff and J. Isidoro. "Active Blobs". In Proc. ICCV 98, January 1998 (to appear).

[ Active Blobs Home | Contact | IVC Home ]

© 1997 Image and Video Computing Group - Boston University 
Last Modified: Oct 27, 1997