Face Reconstruction in the Wild
  ICCV 2011

(a) A few example photos (out of hundreds) we used for the reconstruction of George W. Bush and our reconstruction. (b) Photos of Bill Clinton and our reconstruction. Note the significant variability in facial expression, lighting and pose.	Rendering of the shapes in different viewpoints and photos (not used in reconstruction) of the person in similar viewpoints.

We address the problem of reconstructing 3D face models from large unstructured photo collections, e.g., obtained by Google image search or from personal photo collections in iPhoto. This problem is extremely challenging due to the high degree of variability in pose, illumination, facial expression, non-rigid changes in face shape and reflectance over time and occlusions. In light of this extreme variability, no single reconstruction can be consistent with all of the images. Instead, we define as the goal of reconstruction to recover a model that is locally consistent with the image set. I.e., each local region of the model is consistent with a large set of photos, resulting in a model that captures the dominant trends in the input data for different parts of the face. Our approach leverages multi-image shading, but unlike traditional photometric stereo approaches, allows for changes in viewpoint and shape. We optimize over pose, shape, and lighting in an iterative approach that seeks to minimize the rank of the transformed images. This approach produces high quality shape models for a wide range of celebrities from photos available on the Internet.

Team:

Ira Kemelmacher-Shlizerman - University of Washington
Steve Seitz - University of Washington & Google

Ira Kemelmacher-Shlizerman and Steven M. Seitz. "Face Reconstruction in the Wild." International Conference on Computer Vision (ICCV), Nov 2011.

This work was supported in part by National Science Foundation grant IIS-0811878, the University of Washington Animation Research Labs, Adobe, Google, and Microsoft. In the paper we used publicly available photos of: George W. Bush, Bill Clinton, Tom Hanks, George Clooney, Li Zhang, Colin Powell, Tony Blair, Donald Rumsfeld and Kevin Spacey, as well as photos of Eli Shlizerman from our personal photo collection.

Questions? Please contact Ira Kemelmacher-Shlizerman

Also, check out our related works: "Exploring Photobios" and "Being John Malkovich"!