Environment Matting and Compositing

Douglas Zongker¹ Dawn Werner¹ Brian Curless¹ David Salesin^1,2

¹University of Washington ²Microsoft Research

Abstract
This paper introduces a new process, environment matting, which captures not just a foreground object and its traditional opacity matte from a real-world scene, but also a description of how that object refracts and reflects light, which we call an environment matte. The foreground object can then be placed in a new environment, using environment compositing, where it will refract and reflect light from that scene. Objects captured in this way exhibit not only specular but glossy and translucent effects, as well as selective attenuation and scattering of light according to wavelength. Moreover, the environment compositing process, which can be performed largely with texture mapping operations, is fast enough to run at interactive speeds on a desktop PC. We compare our results to photos of the same objects in real scenes. Applications of this work include the relighting of objects for virtual and augmented reality, more realistic 3D clip art, and interactive lighting design.

Paper

SIGGRAPH 1999 paper (1.7MB PDF)

Citation (bibTex)
Douglas E. Zongker, Dawn M. Werner, Brian Curless, and David H. Salesin. Environment Matting and Compositing. In Proceedings of ACM SIGGRAPH 99, pages 205-214, August 1999

Results

Here are some (very large) QuickTime movies of environment compositing in action.

Goblet of water (9,153K)
Hurricane lamp (8,920K)
Rough-surfaced bookend (8,941K)
Morph 1 (5,767K)
Morph 2 (same objects, different background) (5,669K)