Dynamic Occluding Contours
A New External-Energy Term for Snakes

Michele Covell and Trevor Darrell

Motivation

Object tracking is an important tool for rotoscoping. Tracking using snakes is one obvious approach to the rotoscoping problem: The snake tracks the object outline and then determines the alpha-channel values. However, without detailed object-shape and -motion models, current snake formulations do not reliably track occlusions in natural scenes.

Approach

We have developed a new external-energy term for snakes which allows occlusion tracking without detailed object models. Our paper shows how our image model improves tracking in cluttered scenes. Finally, our paper describes how a high-quality rotoscoping mask is created using our image model both to drive the snake and to refine the silhouette of the snake track to match the object.

For a copy of our paper, please see either the PDF version (440k) or the Postscript version (14M) .

Results

Tracking Boundaries

Our new external-energy term for snakes, called radial cumulative similarity (RCS), captures foreground attributes, as does patch matching; captures the occlusion structure, as does edge matching; and ignores background attributes, as does robust matching.

Snake tracking the inner boundary of the lips.
This sequence is difficult for traditional snake tracking since the lips themselves are low contrast and since the contrast sign of the inner boundary changes, depending on the visibility of the teeth. RCS snake tracking is shown here. 		Snake tracking an inch worm on a cluttered, dynamic background
This sequence is difficult for traditional snake tracking since the background is cluttered with numerous high-contrast edges and since the appearance of the background near the occlusion boundary change dramatically from one frame to the next, especially at the end of the movie. Snake tracking using RCS profiles are shown here. (The last two frames of this movie show drastically different backgrounds. These frames were inserted to test our tracking results under extreme background changes.)

Automatic Rotoscoping


RCS snakes, along with their associated RCS profiles, can be used for semi-automatic rotoscoping of foreground objects. The artist draws a closed-contour snake in the first frame of the sequence. The full length of the snake should lie completely inside the object's occluding contour. This closed-contour snake is propagated through the entire sequence using RCS snake tracking. Once this is done, the selected object is automatically rotoscoped out of the sequence. This is done using a continuous-valued alpha-channel sequence created from the snake position and from the RCS profiles along the snake.

The separate pieces from automatic rotoscoping the inch worm and the lip sequences are available: the inch worm and its surroundings are separated, as are the lips and the inside of the mouth.