Collection Flow
CVPR 2012

Ira Kemelmacher-Shlizerman - University of Washington
Steve Seitz - University of Washington & Google


Computing optical flow between any pair of Internet face photos is challenging for most current state of the art flow estimation methods due to differences in illumination, pose, and geometry. We show that flow estimation can be dramatically improved by leveraging a large photo collection of the same (or similar) object. In particular, consider the case of photos of a celebrity from Google Image Search. Any two such photos may have different facial expression, lighting and face orientation. The key idea is that instead of computing flow directly between the input pair (I,J), we compute versions of the images (I',J') in which facial expressions and pose are normalized while lighting is preserved. This is achieved by iteratively projecting each photo onto an appearance subspace formed from the full photo collection. The desired flow is obtained through concatenation of flows (I-->I') (J'-->J). Our approach can be used with any two-frame optical flow algorithm, and significantly boosts the performance of the algorithm by providing invariance to lighting and shape changes.


Given a pair of images (first and last in the sequence) the in-between photos are automatically synthesized using our flow estimation method. Note the significant variation in lighting and facial expression between the two input photos.


Files:

  Paper 
  Video:



Citation:

Ira Kemelmacher-Shlizerman and Steven M. Seitz. "Collection Flow." IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2012.


Bibtex:
@inproceedings{kemelmacher2012collection,
  title={Collection flow},
  author={Kemelmacher-Shlizerman, Ira and Seitz, Steven M},
  booktitle={Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on},
  pages={1792--1799},
  year={2012},
  organization={IEEE}
}

Acknowledgement:

This work was supported in part by National Science Foundation grant IIS-0811878, the University of Washington Animation Research Labs, Adobe, Google, and Microsoft.

Check out our related works!

- "Exploring Photobios", SIGGRAPH'11
- "Face Reconstruction in the Wild", ICCV'11
- "Being John Malkovich": Ever wanted to be someone else? Now you can! , ECCV'10
- Recent works appear here


Questions? Please contact Ira Kemelmacher-Shlizerman

This page was translated to Russian (by Starmoz) and Bosnian

Number of people visited this page: