Often when we review our holiday photos, we notice things we wish we could have avoided, such as vehicles, construction work, or simply other tourists. We cannot go back and retake the photo, so what can we do if we want to remove these things from our photos? We want to replace these sections of the image in a convincing way, preferably with what would really have been seen there without the occlusions. Previous work on this problem, often referred to as “inpainting”, are mainly applicable to small image regions, and rely largely on models of the local behaviour of natural images, e.g. [3, 5]. Recently, replacing large occlusions in photographs has been approached using images from the Internet [2, 7] or by combining several images captured at approximately the same time [1, 9]. In this paper we leverage recent advances in viewpoint invariant image search [8] to find other images of the same scene on the Internet. Beginning with a query image containing a target region to be replaced, we first use an online image search engine to retrieve images of the same scene, and take these to be a set of oracles. Since these images may have significant variations in viewpoint and lighting, we register each oracle to the query image using multiple homographies and a simple global photometric correction. We then use each oracle to propose a solution, by copying image data into the target region using Poisson blending. Finally, we use a Markov random field (MRF) formulation to combine the proposals into a single, occlusion-free result. Figure 1 shows the main stages of our system, and compares our result to those of two other methods.

/pdf/get-out-of-my-picture-internet-based-inpainting-g5any931cc.pdf

Get Out of my Picture! Internet-based Inpainting.

We introduce an approach for synthesizing time-lapse videos of popular landmarks from large community photo collections. The approach is completely automated and leverages the vast quantity of photos available online. First, we cluster 86 million photos into landmarks and popular viewpoints. Then, we sort the photos by date and warp each photo onto a common viewpoint. Finally, we stabilize the appearance of the sequence to compensate for lighting effects and minimize flicker. Our resulting time-lapses show diverse changes in the world's most popular sites, like glaciers shrinking, skyscrapers being constructed, and waterfalls changing course.

Time-lapse mining from internet photos

Previous works on image completion typically aim to produce visually plausible results rather than factually correct ones. In this paper, we propose an approach to faithfully complete the missing regions of an image. We assume that the input image is taken at a well-known landmark, so similar images taken at the same location can be easily found on the Internet. We first download thousands of images from the Internet using a text label provided by the user. Next, we apply two-step filtering to reduce them to a small set of candidate images for use as source images for completion. For each candidate image, a co-matching algorithm is used to find correspondences of both points and lines between the candidate image and the input image. These are used to find an optimal warp relating the two images. A completion result is obtained by blending the warped candidate image into the missing region of the input image. The completion results are ranked according to combination score, which considers both warping and blending energy, and the highest ranked ones are shown to the user. Experiments and results demonstrate that our method can faithfully complete images.

/pdf/faithful-completion-of-images-of-scenic-landmarks-using-546l3vsjc5.pdf

Faithful Completion of Images of Scenic Landmarks Using Internet Images

A method and system for creating and using a virtual dressing room, by superimposing a non-linearly stretchable object image onto a base image in a display screen of a communication device, the images being planar. The method comprises scanning an encoded indicium associated with the object image, accessing and uploading a URL associated with the object image, the object image including a plurality of object image critical points, accessing the base image at the communication device, the base image including a plurality of base image critical points respectively corresponding to the object image critical points, re-mapping the object image via global transformation of coordinates associated with the object image critical points, such that the re-mapped object image critical points coincide with the respective base image critical points, and superimposing the re-mapped object image onto the base image for display at the display screen.

Method and system for simulating superimposition of a non-linearly stretchable object upon a base object using representative images

The design of computer-assisted decision (CAD) systems for different biomedical imaging scenarios is a challenging task in computer vision. Sometimes, this challenge can be attributed to the image acquisition mechanisms since the lack of control on the cameras can create different visualizations of the same imaging site under different rotation, scaling, and illumination parameters, with a requirement to get a consistent diagnosis by the CAD systems. Moreover, the images acquired from different sites have specific colors, making the use of standard color spaces highly redundant. In this paper, we propose to tackle these issues by introducing novel region-based texture, and color descriptors. The proposed texture features are based on the usage of analytic Gabor filters (for compensation of illumination variations) followed by the calculation of first- and second-order statistics of the filter responses and making them invariant using some trivial mathematical operators. The proposed color features are obtained by compensating for the illumination variations in the images using homomorphic filtering followed by a bag-of-words approach to obtain the most typical colors in the images. The proposed features are used for the identification of cancer in images from two distinct imaging modalities, i.e., gastroenterology and dermoscopy . Experiments demonstrate that the proposed descriptors compares favorably to several other state-of-the-art methods, elucidating on the effectiveness of adapted features for image characterization.

Content-Adaptive Region-Based Color Texture Descriptors for Medical Images

A projector includes: an image capturing unit that captures an image of a projection target image (an original image or a corrected image) projected by the projector on a projection surface; a shape model hold unit that holds shape models; a model selection unit that selects a projection surface shape model based on correspondence relationships between first feature points on the projection target image and second feature points on the captured image; a parameter calculation unit that calculates a model parameter which is to be used to approximate a shape of the projection surface shape model to the shape of the projection surface and indicates a positional relationship between the projection surface and the projector, for each projection target image based on the correspondence relationships; a correction unit that corrects another original image; and a projection unit that projects the corrected image as another projection target image.

Projector and projection method

In this paper, we propose an image completion algorithm which takes advantage of the countless number of images available on Internet photo sharing sites to replace occlusions in an input image. The algorithm 1) automatically selects the most suitable images from a database of downloaded images and 2) seamlessly completes the input image using the selected images with minimal user intervention. Experimental results on input images captured at various locations and scene conditions demonstrate the effectiveness of the proposed technique in seamlessly reconstructing user-defined occlusions.

An Image Completion Algorithm Using Occlusion-Free Images from Internet Photo Sharing Sites

This paper proposes a geometric correction method for projected images using SIFT feature points. A projected image correction method based on projector-camera system needs to obtain the correspondence between projector and camera images, which is obtained by projecting the special patterns on a screen. The special patterns must be re-projected when changing the environment, e.g., the projector or screen is moved. On the other hand, this paper proposes a geometric correction method using SIFT feature points on a projected image which does not employ any special patterns to obtain the correspondence between projector and camera images. Experimental result shows that the use of our proposed method makes it possible to correct distortion of projected images with only one snapshot.

A geometric correction method for projected images using SIFT feature points

This paper proposes an accurate ultrasound probe localization method for 3D US image reconstruction. The proposed method consists of (i) feature tracking of a video sequence and (ii) camera pose estimation using structure from motion (SfM). SfM is used to reconstruct 3D point clouds from multiple-view images and simultaneously estimate each camera position. To apply SfM to a video sequence, the accurate method is required to track features between adjacent frames. We employ a sub-pixel image matching technique using Phase-Only Correlation (POC) for feature tracking. POC is a technique of image matching using phase components in the Fourier transforms of images. Through a set of experiments, we demonstrate that the proposed method can estimate the location of the US probe with about 2mm error for the probe travel distance of 150–200mm.

Probe localization using structure from motion for 3D ultrasound image reconstruction

This paper proposes a high-accuracy image correction method using SIFT (Scale-Invariant Feature Transform) and POC (Phase-Only Correlation) for multi-projector display. The accurate correspondence between the projector and camera images is required to achieve seamless imagery in a multiprojector display. The conventional methods need to project and take special light patterns on a screen many times to obtain the correspondence. On the other hand, the proposed method needs to take only one snapshot of ordinary images so as to realize real-time geometric correction of projector images. Through a set of experiments, we demonstrate that the proposed method is effective for practical use of multi-projector display compared with the conventional methods.

Satoshi Kondo

Papers

Projector and projection method

An Image Completion Algorithm Using Occlusion-Free Images from Internet Photo Sharing Sites

A geometric correction method for projected images using SIFT feature points

Probe localization using structure from motion for 3D ultrasound image reconstruction

Performance evaluation of a geometric correction method for multi-projector display using SIFT and Phase-Only Correlation