Digital Matting and Compositing

Overview

Matting and compositing are important operations in the production of special effects. These techniques enable directors to embed actors in a world that exists only in imagination, or to revive creatures that have been extinct for millions of years. During matting, foreground elements are extracted from a film or video sequence. During compositing, the extracted foreground elements are placed over novel background images.

Traditional approaches to matting include blue-screen matting and rotoscoping. The former requires filming in front of an expensive blue screen under carefully controlled lighting, and the latter demands talent and intensive user interaction. Our Bayesian matting algorithm (CVPR 2001) can pull alpha mattes of complex shapes from natural images. In the "video matting" paper (SIGGRAPH 2002), we extended this approach to video by interpolating user-drawn keyframes using optical flow. A novel technique for smoke matte extraction is also demonstrated.

Traditional compositing operations can only model color blending effects like anti-aliasing, motion blur and transparency. This model, however, can't model reflections, refractions and shadows. In SIGGRAPH 1999, we introduced environment matting which can captures how a foreground object refracts and reflects light. The foreground object can then be placed in a new environment using environment compositing, where it will refract and reflect light from that scene. We later developed more sophisticated sampling schemes to capture mattes with higher accuracies, and techniques requiring fewer images, to allow for real-time capture. Shadows are yet another effects that traditional approaches fail to model correctly. In SIGGRAPH 2003, we introduced a novel process called "shadow matting and compositing" to acquire the photometric and geometric properties of the background for making realistic shadow composites.

Personnel

Publications

Shadow Matting and Compositing
Yung-Yu Chuang, Dan B Goldman, Brian Curless, David Salesin, Richard Szeliski
Proc. ACM SIGGRAPH 2003
Video Matting of Complex Scenes
Yung-Yu Chuang, Aseem Agarwala, Brian Curless, David Salesin, Richard Szeliski
Proc. ACM SIGGRAPH 2002
A Bayesian Approach to Digital Matting
Yung-Yu Chuang, Brian Curless, David Salesin, Richard Szeliski
Proc. CVPR 2001
Environment Matting Extensions: Towards Higher Accuracy and Real-Time Capture
Yung-Yu Chuang, Douglas Zongker, Joel Hindorff, Brian Curless, David Salesin, Richard Szeliski
Proc. ACM SIGGRAPH 2000
Environment Matting and Compositing
Douglas Zongker, Dawn Werner, Brian Curless, David Salesin
Proc. ACM SIGGRAPH 1999

Support

This research is supported by:
cyy -a-t- cs.washington.edu