Active Blobs: Examples of Nonrigid Shape Tracking

The active blobs system has been implemented on a SGI Indigo2 Impact with an R10K processor running at 195 MHz with 192 MB of RAM. This workstation provides texture map acceleration in hardware. All performance statistics are reported for unoptimized code. Experiments were conducted using both the Marquardt-Levenberg and difference decomposition minimization methods, as detailed in the technical report.

Figure 1: On-line tracking of a deformable ball. Marquardt-Levenberg minimization averaged 2 frames/sec, while the difference decomposition averaged 12.4 frames/sec. For visualization purposes, an outline of the active blob is shown overlaid on the original input images. The recovered active blob warps for Marquardt-Levenberg are shown below each input image. Difference decomposition tracked with comparable accuracy.

In the first example, Fig. 1, an active blob is used to track a region on a deforming green ball. This tracking was done on-line. In other words, the system tracked using the current frame taken from live video input. If tracking computation per frame could not keep up with full video frame-rate, then in-between frames were skipped.

Frames from the original video sequence are shown in the top row of Fig. 1. The region contained 2394 pixels, and deformation was modeled using the ten lowest-order modal parameters. The recovered active blob tracking for Marquardt-Levenberg is shown below each input image. Using the Marquardt-Levenberg minimization technique, this example ran at an average of two frames a second, with on average 2 to 3 iterations required for convergence. The precomputation time was 0.13 secs.

The same sequence was successfully tracked using the difference decomposition, at an average rate of 12.4 frames per second. Precomputation of the difference basis and matrix inverses took six seconds.

Figure 2: On-line tracking a deforming plastic bag filled with candy using difference decomposition. This figure shows every twentieth frame in the tracking sequence. For visualization purposes, an outline of the active blob is shown overlaid on the original input images. Marquardt-Levenberg minimization averaged 0.9 frames/sec, while the difference decomposition averaged 8.4 frames/sec. Results for the difference decomposition are shown below each input image. Marquardt-Levenberg was not able to track the object completely through this sequence.

Figure 2 shows another on-line tracking example, this time tracking a bag of candy. The user circled the region of interest, as shown in the upper left hand corner of Fig. 2. The resulting active blob contained 2680 pixels, and deformation was modeled using the twenty lowest-order modal parameters.

The figure shows every twentieth frame in the tracking sequence. The resulting difference decomposition tracking is shown below each input image. Tracking was accomplished at 8.4 frames per second via the difference decomposition. The required precomputation for basis and inverse matrices took 21 CPU seconds. As can be seen, tracking performs well, despite very large deformations and changes in orientation, specular highlights, and moving shadows.

The same sequence was tracked using Marquardt-Levenberg, at an average rate of 0.9 frames per second. Tracking was less successful, diverging part way through the sequence. This problem can be solved when the Marquardt-Levenberg procedure is run off-line, allowing tracking using all frames from the video sequence.

[ Active Blobs Home | Contact | IVC Home ]

Last Modified: Oct 27, 1997