This paper proposes a new Bayesian framework for solving the matting problem, i.e. extracting a foreground element from a background image by estimating an opacity for each pixel of the foreground element. Our approach models both the foreground and background color distributions with spatially-varying mixtures of Gaussians, and assumes a fractional blending of the foreground and background colors to produce the final output. It then uses a maximum-likelihood criterion to estimate the optimal opacity, foreground and background simultaneously. In addition to providing a principled approach to the matting problem, our algorithm effectively handles objects with intricate boundaries, such as hair strands and fur, and provides an improvement over existing techniques for these difficult cases.

Yung-Yu Chuang, Brian Curless, David H. Salesin, and Richard Szeliski. A Bayesian Approach to Digital Matting. In

CVPR 2001 paper (3.6MB PDF)

- We forgot to mention one thing in the paper.
Because foreground and background samples are also observations from the camera,
they should have the same noise characteristics as the observation C.
Hence, we added the same amount of camera variance \sigma
_{c}to the covariance matrices of foreground and background samples in Equation (7). We used eigen-analysis to find the orientation of the covariance matrix and added \sigma_{c}^{2}in every axis. That is, we decomposed \Sigma_{F}as U S V^{T}. Let S=diag{s_{1}^{2},s_{2}^{2},s_{3}^{2}}, we set S'=diag(s_{1}^{2}+\sigma_{c}^{2}, s_{2}^{2}+\sigma_{c}^{2}, s_{3}^{2}+\sigma_{c}^{2}) and assign the new \Sigma_F as U S' V^{T}. By doing so, we also avoided most of the degenerate cases, i.e., non-invertible matrices.

- For the window for collecting foreground and background samples, we set a minimal window size and a minimal number of samples. We start from a window with the minimal window sizw. If such a window does not give us enough samples, we gradually increase the window until the minimal number of samples is satistified. Note that, in this way, the windows for background and foreground might end up with different sizes.

Blue-screen matting | Difference matting | Natural image matting | ||

Input | ||||

Segmentation | ||||

Composite (Bayesian) |

Woman image was obtained from Corel Knockout's tutorial, Copyright © 2001 Yung-Yu Chuang, Brian Curless, David Salesin, Richard Szeliski and its licensors Corel. All rights reserved.

Alpha Matte | Composite (black) | Inset | Composite | |

Mishima | ||||

Bayesian | ||||

Ground truth |

Alpha Matte | Composite | Inset | |

Difference Matting | |||

Knockout | |||

Ruzon & Tomasi | |||

Bayesian | |||

Ground Truth |

Alpha Matte | Composite | Inset | Alpha Matte | Composite | Inset | |

Knockout | ||||||

Ruzon & Tomasi | ||||||

Bayesian |

Input | |||||

Alpha | |||||

Composite |

Woman image was obtained from Corel Knockout's tutorial.

cyy -a-t- cs.washington.edu