Spacetime Faces: High-Resolution Capture for Modeling and Animation Li Zhang, Noah Snavely, Brian Curless, and Steven M. Seitz

Abstract

We present an end-to-end system that goes from video sequences to high resolution, editable, dynamically controllable face models. The capture system employs synchronized video cameras and structured light projectors to record videos of a moving face from multiple viewpoints. A novel spacetime stereo algorithm is introduced to compute depth maps accurately and overcome over-fitting deficiencies in prior work. A new template fitting and tracking procedure fills in missing data and yields point correspondence across the entire sequence without using markers. We demonstrate a data-driven, interactive method for inverse kinematics that draws on the large set of fitted templates and allows for posing new expressions by dragging surface points directly. Finally, we describe new tools that model the dynamics in the input sequence to enable new animations, created via key-framing or texture-synthesis techniques.

Citation (bibTex)
Li Zhang, Noah Snavely, Brian Curless, and Steven M. Seitz. Spacetime Faces: High-resolution capture for modeling and animation. In ACM SIGGRAPH Proceedings, Los Angeles, CA, Aug., 2004, pp. to appear. [Paper: PDF(10.3M),PS.GZ(18.2M)]

Videos shown in SIGGRAPH 2004 presentation

Gray shaded depth map pairs	1016×656 AVI(89M)
Screen capture of FaceIK demo	640×480 AVI(350K)

Videos included in SIGGRAPH 2004 DVD

• Full length:

  320×240 DivX(75M)

  640×480 AVI (97M)

• Segments:

  Capture	640×480 AVI (44M)
  Fitting	640×480 AVI (22M)
  FaceIK	640×480 AVI (8.6M)
  Video Texture	640×480 AVI (7.5M)
  Keyframe Animation	640×480 AVI (14.0M)

In media

• Human Body Measurement Newsletter, Homometrica Consulting, April, 2005.

See my previous work on Shape and Motion under Varying Illumination!

See my previous work on Spacetime Stereo!

See my previous work on Single View Modeling!