Panoramic Video Textures

Aseem Agarwala¹ Colin Zheng¹ Chris Pal³ Maneesh Agrawala² Michael Cohen² Brian Curless¹ David Salesin^1,2 Richard Szeliski²

¹University of Washington ²Microsoft Research ³University of Massachusetts, Amherst

Abstract
This paper describes a mostly automatic method for taking the output of a single panning video camera and creating a panoramic video texture (PVT): a video that has been stitched into a single, wide field of view and that appears to play continuously and indefinitely. The key problem in creating a PVT is that although only a portion of the scene has been imaged at any given time, the output must simultaneously portray motion throughout the scene. Like previous work in video textures, our method employs min-cut optimization to select fragments of video that can be stitched together both spatially and temporally. However, it differs from earlier work in that the optimization must take place over a much larger set of data. Thus, to create PVTs, we introduce a dynamic programming step, followed by a novel hierarchical min-cut optimization algorithm. We also use gradient-domain compositing to further smooth boundaries between video fragments. We demonstrate our results with an interactive viewer in which users can interactively pan and zoom on high-resolution PVTs.

Citation
Aseem Agarwala, Colin Zheng, Chris Pal, Maneesh Agrawala, Michael Cohen, Brian Curless, David Salesin, Richard Szeliski. Panoramic Video Textures. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2005), 2005.

Paper
SIGGRAPH 2005 pre-print (700K PDF)

Video
Video (5 min, 720x480, 112 MB, WMV format)

Using Windows Media Player 10? The video may appear a little dark. Here's how to fix it.

Interactive viewer and data sets
By far, the best way to view our results at their full resolution is within our interactive viewer. Our viewer is a Windows XP application; however, it requires a strong computer with a good graphics card to play the results at their full frame-rate. In our experience, you need at least 1 GB of RAM, and a graphics card with at least 128 MB of texture memory. The waterfall data set is the most intensive, so try one of the others first. Also, the data sets are fairly large; the size of the download is listed in the table below. If you have success on a machine that doesn't meet the above requirements, please let us know!

Viewer (400K Zip file)

Data set name & size	Input video (WMV, 1/2 resolution)	Data Set
Waterfall (104MB)
Yachts (64MB)
Park (105MB)
Waterfront (112MB)

Figure 3, video form
Figure 3 of our paper shows one frame of a comparison between the original video, a result computing using a simple approach, and a result computed using our approach. The full videos of this comparison can be found here. Notice that our approach more faithfully mimics the input, and that it loops seamlessly.

Original video	Simple Approach	Our approach

Errata
We cited the incorrect paper when introducing the alpha-expansion algorithm and when describing the optimality properties of alpha-expansion. In the fourth paragraph of Section 3.5 and the third paragraph of Section 3.6 we should have cited the following paper rather than Kolmogorov and Zabih 2002.

Boykov, Y., Veksler, O., Zabih, R.. Fast Approximate Energy Minimization via Graph Cuts. In IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). vol. 23, no.11, pp.1222-1239, 2001.

Contact Info
aseem@agarwala.org