Spacetime Faces: High-Resolution Capture for Modeling and Animation

Li Zhang, Noah Snavely, Brian Curless, and Steven M. Seitz


We present an end-to-end system that goes from video sequences to high resolution, editable, dynamically controllable face models. The capture system employs synchronized video cameras and structured light projectors to record videos of a moving face from multiple viewpoints. A novel spacetime stereo algorithm is introduced to compute depth maps accurately and overcome over-fitting deficiencies in prior work. A new template fitting and tracking procedure fills in missing data and yields point correspondence across the entire sequence without using markers. We demonstrate a data-driven, interactive method for inverse kinematics that draws on the large set of fitted templates and allows for posing new expressions by dragging surface points directly. Finally, we describe new tools that model the dynamics in the input sequence to enable new animations, created via key-framing or texture-synthesis techniques.

Citation (bibTex)
Li Zhang, Noah Snavely, Brian Curless, and Steven M. Seitz. Spacetime Faces: High-resolution capture for modeling and animation. In ACM SIGGRAPH Proceedings, Los Angeles, CA, Aug., 2004, pp. to appear. [Paper: PDF(10.3M),PS.GZ(18.2M)]

Now the 3D face mesh data is available for research purposes!

Videos shown in SIGGRAPH 2004 presentation
Gray shaded depth map pairs 1016×656 AVI(89M)
Screen capture of FaceIK demo 640×480 AVI(350K)

Videos included in SIGGRAPH 2004 DVD

In media

See my previous work on Shape and Motion under Varying Illumination!

See my previous work on Spacetime Stereo!

See my previous work on Single View Modeling!