Photo Wake-Up: 3D Character Animation from a Single Photo

Chung-Yi Weng1, Brian Curless1, Ira Kemelmacher-Shlizerman1,2
1University of Washington     2Facebook, Inc





Given a single photo as input (far left), we create a 3D animatable version of the subject, which can now walk towards the viewer (middle). The 3D result can be experienced in augmented reality (right); in the result above the user has virtually hung the artwork with a HoloLens headset and can watch the character run out of the painting from different views. Please see all results in the supplementary video.

Supplementary Video




Abstract

We present a method and application for animating a human subject from a single photo. E.g., the character can walk out, run, sit, or jump in 3D. The key contributions of this paper are: 1) an application of viewing and animating humans in single photos in 3D, 2) a novel 2D warping method to deform a posable template body model to fit the person's complex silhouette to create an animatable mesh, and 3) a method for handling partial self occlusions. We compare to state-of-the-art related methods and evaluate results with human studies. Further, we present an interactive interface that allows re-posing the person in 3D, and an augmented reality setup where the animated 3D person can emerge from the photo into the real world. We demonstrate the method on photos, posters, and art.

Paper

Paper (CVPR 2019)     arXiv
Supplementary material


Press & News




Acknowledgments

The authors thank Konstantinos Rematas for helpful discussions, Bogo et al. and Alldieck et al. for sharing their research code, and labmates from UW GRAIL lab for the greatest support. This work was supported by NSF/Intel Visual and Experimental Computing Award #1538618, the UW Reality Lab, Facebook, Google, Huawei, and a Reality Lab Huawei Fellowship.

Last modified: