Spacetime Stereo: Shape Recovery for Dynamic Scenes Li Zhang, Brian Curless, and Steven M. Seitz

Abstract

This paper extends the traditional binocular stereo problem into the spacetime domain, in which a pair of video streams is matched simultaneously instead of matching pairs of images frame by frame. Almost any existing stereo algorithm may be extended in this manner simply by replacing the image matching term with a spacetime term. By utilizing both spatial and temporal appearance variation, this modification reduces ambiguity and increases accuracy. Three major applications for spacetime stereo are proposed in this paper. First, spacetime stereo serves as a general framework for structured light scanning and generates high quality depth maps for static scenes. Second, spacetime stereo is effective for a class of natural scenes, such as waving trees and flowing water, which have repetitive textures and chaotic behaviors and are challenging for existing stereo algorithms. Third, the approach is one of very few existing methods that can robustly reconstruct objects that are moving and deforming over time, achieved by use of oriented spacetime windows in the matching procedure. Promising experimental results in the above three scenarios are demonstrated.

Citation (bibTex)
Li Zhang, Brian Curless, and Steven M. Seitz. Spacetime Stereo: Shape Recovery for Dynamic Scenes. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Madison, WI, June, 2003, pp. 367-374. [Paper: PDF(1.5M); Poster: PDF(1.9M)]

Videos Shown at CVPR 2003

Facial deformation

		=>
Stereo pair video			Spacetime Reconstruction
Left sequence (AVI 6.1M)	Right sequence (AVI 6.0M)

Arm bending

		=>
Stereo pair video			Spacetime Reconstruction
Left sequence (AVI 6.0M)	Right sequence (AVI 6.0M)

Notes:

• A short sequence of stripe patterns are generated randomly and projected periodically in the above experiments.

See my following work on Spacetime Faces!

See my previous work on Rapid Shape Acquistion!