Abstract
We present a framework for automatically reconfiguring images of street scenes by populating, depopulating, or repopulating them with objects such as pedestrians or vehicles. Applications of this method include anonymizing images to enhance privacy, generating data augmentations for perception tasks like autonomous driving, and composing scenes to achieve a certain ambiance, such as empty streets in the early morning. At a technical level, our work has three primary contributions: (1) a method for clearing images of objects, (2) a method for estimating sun direction from a single image, and (3) a way to compose objects in scenes that respects scene geometry and illumination. Each component is learned from data with minimal ground truth annotations, by making creative use of large-numbers of short image bursts of street scenes. We demonstrate convincing results on a range of street scenes and illustrate potential applications.
Paper
Main paper:
[pdf, 3.9 MB]
Supplementary material:
[pdf, 3.1 MB]
arXiv:
[Link]
@InProceedings{wang2021repopulating,
title={Repopulating Street Scenes},
author={Wang, Yifan and Liu, Andrew and Tucker, Richard and Wu, Jiajun and Curless, Brian L and Seitz, Steven M and Snavely, Noah},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2021}
}
Acknowledgments
This work was done while Yifan was interning at Google Research.