Showing papers by "Jian Sun published in 2005"

Image completion with structure propagation

Abstract: In this paper, we introduce a novel approach to image completion, which we call structure propagation. In our system, the user manually specifies important missing structure information by extending a few curves or line segments from the known to the unknown regions. Our approach synthesizes image patches along these user-specified curves in the unknown region using patches selected around the curves in the known region. Structure propagation is formulated as a global optimization problem by enforcing structure and consistency constraints. If only a single curve is specified, structure propagation is solved using Dynamic Programming. When multiple intersecting curves are specified, we adopt the Belief Propagation algorithm to find the optimal patches. After completing structure propagation, we fill in the remaining unknown regions using patch-based texture synthesis. We show that our approach works well on a number of examples that are challenging to state-of-the-art techniques.

Symmetric stereo matching for occlusion handling

Abstract: In this paper, we propose a symmetric stereo model to handle occlusion in dense two-frame stereo. Our occlusion reasoning is directly based on the visibility constraint that is more general than both ordering and uniqueness constraints used in previous work. The visibility constraint requires occlusion in one image and disparity in the other to be consistent. We embed the visibility constraint within an energy minimization framework, resulting in a symmetric stereo model that treats left and right images equally. An iterative optimization algorithm is used to approximate the minimum of the energy using belief propagation. Our stereo model can also incorporate segmentation as a soft constraint. Experimental results on the Middlebury stereo images show that our algorithm is state-of-the-art.

Video object cut and paste

Abstract: In this paper, we present a system for cutting a moving object out from a video clip. The cutout object sequence can be pasted onto another video or a background image. To achieve this, we first apply a new 3D graph cut based segmentation approach on the spatial-temporal video volume. Our algorithm partitions watershed presegmentation regions into foreground and background while preserving temporal coherence. Then, the initial segmentation result is refined locally. Given two frames in the video sequence, we specify two respective windows of interest which are then tracked using a bi-directional feature tracking algorithm. For each frame in between these two given frames, the segmentation in each tracked window is refined using a 2D graph cut that utilizes a local color model. Moreover, we provide brush tools for the user to control the object boundary precisely wherever needed. Based on the accurate binary segmentation result, we apply coherent matting to extract the alpha mattes and foreground colors of the object.

An adaptive MLS surface for reconstruction with guarantees

Abstract: Recent work have shown that moving least squares (MLS) surfaces can be used effectively to reconstruct surfaces from possibly noisy point cloud data. Several variants of MLS surfaces have been suggested, some of which have been analyzed theoretically for guarantees. These analyses, so far, have assumed uniform sampling density. We propose a new variant of the MLS surface that, for the first time, incorporates local feature sizes in its formulation, and we analyze it for reconstruction guarantees using a non-uniform sampling density. The proposed variant of the MLS surface has several computational advantages over existing MLS methods.

Normal estimation for point clouds: a comparison study for a Voronoi based method

Abstract: Many applications that process a point cloud data benefit from a reliable normal estimation step. Given a point cloud presumably sampled from an unknown surface, the problem is to estimate the normals of the surface at the data points. Two approaches, one based on numerical optimizations and another based on Voronoi diagrams are known for the problem. Variations of numerical approaches work well even when point clouds are contaminated with noise. Recently a variation of the Voronoi based method is proposed for noisy point clouds. The centrality of the normal estimation step in point cloud processing begs a thorough study of the two approaches so that one knows which approach is appropriate for what circumstances. This paper presents such results.

Symmetric stereo model for handling occlusion

Abstract: The present symmetric stereo matching technique provides a method for iteratively estimating a minimum energy for occlusion and disparity using belief propagation. The minimum energy is based on an energy minimization framework in which a visibility constraint is embedded. By embedding the visibility constraint, the present symmetric stereo matching technique treats both images equally, instead of treating one as a reference image. The visibility constraint ensures that occlusion in one view and the disparity in another view are consistent.

Bidirectional tracking using trajectory segment analysis

Abstract: In this paper, we present a novel approach to keyframe-based tracking, called bi-directional tracking. Given two object templates in the beginning and ending keyframes, the bi-directional tracker outputs the MAP (maximum a posterior) solution of the whole state sequence of the target object in the Bayesian framework. First, a number of 3D trajectory segments of the object are extracted from the input video, using a novel trajectory segment analysis. Second, these disconnected trajectory segments due to occlusion are linked by a number of inferred occlusion segments. Last, the MAP solution is obtained by trajectory optimization in a coarse-to-fine manner. Experimental results show the robustness of our approach with respect to sudden motion, ambiguity, and short and long periods of occlusion.

Poisson matting for images

Abstract: An exemplary method includes receiving a trimap for an image that specifies a background region, a foreground region and an unknown region for the image wherein a boundary exists between the foreground region and the unknown region and wherein another boundary exists between the unknown region and the background region, solving a set of Poisson equations having boundary conditions for the two boundaries to provide a matte that distinguishes a foreground region from a background region in the unknown region, and refining the matte by solving a set of Poisson equations for a local unknown region. Various other exemplary technologies are also presented.

Systems and methods for image data separation

Abstract: A method includes receiving a first set of one or more data nodes specified by a user using a first specification mode, receiving a second set of one or more data nodes specified by a user using a second specification mode, and automatically identifying a data node to be separated from a collection of data nodes based on a similarity measure characterizing similarity between the data node to be separated and the one or more data nodes in the first set and the one or more data nodes in the second set. A system includes an image processing module automatically segmenting a determined region from an image based on a similarity measure characterizing similarity between pixels in the determined region and a set of one or more specified seed pixels associated with pixels to be included in the determined region.

Image data separating system and method

Abstract: PROBLEM TO BE SOLVED: To provide an image data separating system and its method. SOLUTION: This method comprises receiving a first set of data node designated by a user using a first designation mode, receiving a second set of data node designated by a user using a second designation mode, and automatically distinguishing the data node separated from the set of the data node on the basis of similarity criterion between the data node separated from the set of the data node and the data node of the first set and the data node of the second set. This system comprises an image processing module for automatically sectioning a determined area from the image on the basis of similarity criterion between an image element in the determined area and the set of one or more designated seed image elements related to the image element to be included in the determined area. COPYRIGHT: (C)2006,JPO&NCIPI

Separating a video object from a background of a video sequence

Abstract: Video object cutting and pasting is described. In one implementation, pre-segmentation of video frames into regions is performed prior to a 3-D graph cut segmentation. The 3-D graph cut segmentation uses temporal coherence and a global color model to achieve accuracy of video object boundaries. A 2-D local graph cut segmentation can then be used to refine the boundaries. The boundaries can be tracked within a user-selected sequence of windows and refined using a local color model.