Interactive Digital Photomontage


Seldom does a photograph record what we perceive with our eyes. Often, the scene captured in a photo is quite unexpected — and disappointing — compared to what we believe we have seen. A common example is catching someone with their eyes closed: we almost never consciously perceive an eye blink, and yet, there it is in the photo — “the camera never lies.” Our higher cognitive functions constantly mediate our perceptions so that in photography, very often, what you get is decidedly not what you perceive. “What you get,” generally speaking, is a frozen moment in time, whereas “what you perceive” is some time- and spatially-filtered version of the evolving scene. In this work, we look at how digital photography can be used to create photographic images that more accurately convey our subjective impressions—or go beyond them, providing visualizations or a greater degree of artistic expression. Our approach is to utilize multiple photos of a scene, taken with a digital camera, in which some aspect of the scene or camera parameters varies with each photo. These photographs are then pieced together, via an interactive system, to create a single photograph that better conveys the photographer’s subjective perception of the scene. We call this process digital photomontage, after the traditional process
of combining parts of a variety of photographs to form a composite picture, known as photomontage.

We encourage you to view the video as an introduction to the framework. The paper is also a helpful guide for the types of effect possible.

Interaction Paradigms

 The user usually works with two windows: the composite window and the source window.  The composite window, on the left,  shows the current composite and the source window, on the right, shows the set of images that can be combined to form the composite. The user can scroll through the set of images using the number dial in the source window. For most applications the order of the images in the source window is of no consequence.



The user can form the composite using a series of objectives. The types of objectives currently in the framework include color, max/min luminance, max/min likelihood, max/min contrast, etc. These objectives can be applied globally to the whole composite, such as when one wants to create an extended depth-of-field image from a set of images with different focal lengths. Or they can be applied locally through a painting style interface. The user paints with rough brush strokes and the framework is able to combine the desired characteristics into the composite as seamlessly as possible. For specific examples, please look through the tutorials. When interacting with the composite locally through painting, the user can paint with one of two brushes, the single-image brush and the multi-image brush.

Single-image Brush

With the single-image brush the user wishes to add just one image to the current composite, and that that image should be the best one that both satisfies the objective under the stroke as well as fits with the existing composite as seamlessly as possible. To provide the user control over the choice of this best image, immediately after painting the user is shown a third window, called the selection window.



The selection window provides a set of images that best fit the specificied objective under the stroke. Each image has a region masked in blue which specifies the cut that would be included in the composite if the user was to choose that image. In the visualization above, the user is viewing the 3rd best image while trying to eliminate a shadow on the wall. The composite window shows how that blue region fits with the existing composite. The user can scroll through all the potential cuts and pick one (the cut is already computed) or refine it by painting additional strokes.


The additional strokes remove the whole shadow.



Once the user is satisfied, he can accept this new addition to the composite with the "Take this cut button" or reject it and start over again.

Multi-image Brush

The multi-image brush is useful when one image does not include all the required effects. As an example, to create the halo around the subject's head, we used just one stroke of the multi-image brush . In such cases the framework proceeds as if a global objective had been specified: all possible source images are fused together in order to best meet the requirements of the locally-specified image objective, in this case the max luminance objective. However, this operation can take significant time to compute.



Description of Menus

Composite Window






Source Window

Selection Window

Tutorials:

  1. Improving Family Portraits