Representations for interactive photorealistic visualization of scenes range from compact 2D panoramas to data-intensive 4D light fields.
In this paper, we propose a technique for creating a layered representation
from a sparse set of images taken with a hand-held camera. This representation,
which we call a layered depth panorama (LDP),
allows the user to experience 3D by off-axis panning. It combines the
compelling experience of panoramas with limited 3D navigation. Our choice
of representation is motivated by ease of capture and compactness. We
formulate the problem of constructing the LDP
as the recovery of color and geometry in a multi-perspective cylindrical
disparity space. We leverage a graph cut approach to sequentially determine
the disparity and color of each layer using multi-view stereo. Geometry
visible through the cracks at depth discontinuities in a frontmost layer
is determined and assigned to layers behind the frontmost layer. All layers
are then used to render novel panoramic views with parallax. We demonstrate
our approach on a variety of complex outdoor and indoor scenes.
Ke Colin Zheng, Sing Bing Kang, Michael Cohen, and Richard Szeliski. Layered Depth Panoramas. Proceedings of CVPR, 2007.
CVPR 2007 pre-print (4MB PDF)
Click images on the top to view the results step by step. For each example, we show the input sequence, a 2D visualization of the scene and the camera setup, and the depth map and the texture map for each layer of LDP. All the intermediate results can be viewed at its full resolutioin by clicking them. Note we use two different parameterizations to represent the 3D space: either from a single camera sitting at the center of the cylinder, or from cameras along the circular arc. We demonstrates our technique using both representations. With LDP, we now can pan the camera around, move from side to side, go forward and backward, zoom in and out, and perceive certain amount of parallax. Videos are provided to demonstrate such viewing experience. Just click the "viewing experience" links in the page to enjoy!