Showing papers on "Real image published in 2003"

Simultaneous structure and texture image inpainting

Abstract: An algorithm for the simultaneous filling-in of texture and structure in regions of missing image information is presented in this paper. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture filling-in algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filled-in with texture synthesis techniques. The original image is then reconstructed adding back these two sub-images. The novel contribution of this paper is then in the combination of these three previously developed components, image decomposition with inpainting and texture synthesis, which permits the simultaneous use of filling-in algorithms that are suited for different image characteristics. Examples on real images show the advantages of this proposed approach.

Object removal by exemplar-based inpainting

Abstract: A new algorithm is proposed for removing large objects from digital images. The challenge is to fill in the hole that is left behind in a visually plausible way. In the past, this problem has been addressed by two classes of algorithms: (i) "texture synthesis" algorithms for generating large image regions from sample textures, and (ii) "inpainting" techniques for filling in small image gaps. The former work well for "textures" - repeating two dimensional patterns with some stochasticity; the latter focus on linear "structures" which can be thought of as one dimensional patterns, such as lines and object contours. This paper presents a novel and efficient algorithm that combines the advantages of these two approaches. We first note that exemplar-based texture synthesis contains the essential process required to replicate both texture and structure; the success of structure propagation, however, is highly dependent on the order in which the filling proceeds. We propose a best-first algorithm in which the confidence in the synthesized pixel values is propagated in a manner similar to the propagation of information in inpainting. The actual color values are computed using exemplar-based synthesis. Computational efficiency is achieved by a block-based sampling process. A number of examples on real and synthetic images demonstrate the effectiveness of our algorithm in removing large occluding objects as well as thin scratches. Robustness with respect to the shape of the manually selected target region is also demonstrated. Our results compare favorably to those obtained by existing techniques.

Simultaneous structure and texture image inpainting

Abstract: An algorithm for the simultaneous filling-in of texture and structure in regions of missing image information is presented. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture filling-in algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filled-in with texture synthesis techniques. The original image is then reconstructed adding back these two sub-images. The novel contribution of the paper is then in the combination of these three previously developed components: image decomposition with inpainting and texture synthesis, which permits the simultaneous use of filling-in algorithms that are suited for different image characteristics. Examples on real images show the advantages of this proposed approach.

An Efficient Image Similarity Measure Based on Approximations of KL-Divergence Between Two Gaussian Mixtures

Abstract: In this work we present two new methods for approximating the Kullback-Liebler (KL) divergence between two mixtures of Gaussians. The first method is based on matching between the Gaussian elements of the two Gaussian mixture densities. The second method is based on the unscented transform. The proposed methods are utilized for image retrieval tasks. Continuous probabilistic image modeling based on mixtures of Gaussians together with KL measure for image similarity, can be used for image retrieval tasks with remarkable performance. The efficiency and the performance of the KL approximation methods proposed are demonstrated on both simulated data and real image datasets. The experimental results indicate that our proposed approximations outperform previously suggested methods.

Shape context and chamfer matching in cluttered scenes

Abstract: This paper compares two methods for object localization from contours: shape context and chamfer matching of templates. In the light of our experiments, we suggest improvements to the shape context: shape contexts are used to find corresponding features between model and image. In real images it is shown that the shape context is highly influenced by clutters; furthermore, even when the object is correctly localized, the feature correspondence may be poor. We show that the robustness of shape matching can be increased by including a figural continuity constraint. The combined shape and continuity cost is minimized using the Viterbi algorithm on features, resulting in improved localization and correspondence. Our algorithm can be generally applied to any feature based shape matching method. Chamfer matching correlates model templates with the distance transform of the edge image. This can be done efficiently using a coarse-to-fine search over the transformation parameters. The method is robust in clutter, however, multiple templates are needed to handle scale, rotation and shape variation. We compare both methods for locating hand shapes in cluttered images, and applied to word recognition in EZ-Gimpy images.

An efficient image similarity measure based on approximations of KL-divergence between two gaussian mixtures

Abstract: We present two new methods for approximating the Kullback-Liebler (KL) divergence between two mixtures of Gaussians. The first method is based on matching between the Gaussian elements of the two Gaussian mixture densities. The second method is based on the unscented transform. The proposed methods are utilized for image retrieval tasks. Continuous probabilistic image modeling based on mixtures of Gaussians together with KL measure for image similarity, can be used for image retrieval tasks with remarkable performance. The efficiency and the performance of the KL approximation methods proposed are demonstrated on both simulated data and real image data sets. The experimental results indicate that our proposed approximations outperform previously suggested methods.

Inferring 3D structure with a statistical image-based shape model

Abstract: We present an image-based approach to infer 3D structure parameters using a probabilistic "shape+structure" model. The 3D shape of an object class is represented by sets of contours from silhouette views simultaneously observed from multiple calibrated cameras, while structural features of interest on the object are denoted by a number of 3D locations. A prior density over the multiview shape and corresponding structure is constructed with a mixture of probabilistic principal components analyzers. Given a novel set of contours, we infer the unknown structure parameters from the new shape's Bayesian reconstruction. Model matching and parameter inference are done entirely in the image domain and require no explicit 3D construction. Our shape model enables accurate estimation of structure despite segmentation errors or missing views in the input silhouettes, and it works even with only a single input view. Using a training set of thousands of pedestrian images generated from a synthetic model, we can accurately infer the 3D locations of 19 joints on the body based on observed silhouette contours from real images.

Image repairing: robust image synthesis by adaptive ND tensor voting

Abstract: We present a robust image synthesis method to automatically infer missing information from a damaged 2D image by tensor voting. Our method translates image color and texture information into an adaptive ND tensor, followed by a voting process that infers non-iteratively the optimal color values in the ND texture space for each defective pixel. ND tensor voting can be applied to images consisting of roughly homogeneous and periodic textures (e.g. a brick wall), as well as difficult images of natural scenes, which contain complex color and texture information. To effectively tackle the latter type of difficult images, a two-step method is proposed. First, we perform texture-based segmentation in the input image, and extrapolate partitioning curves to generate a complete segmentation for the image. Then, missing colors are synthesized using ND tensor voting. Automatic tensor scale analysis is used to adapt to different feature scales inherent in the input. We demonstrate the effectiveness of our approach using a difficult set of real images.

Illumination from shadows

Abstract: In this paper, we introduce a method for recovering an illumination distribution of a scene from image brightness inside shadows cast by an object of known shape in the scene. In a natural illumination condition, a scene includes both direct and indirect illumination distributed in a complex way, and it is often difficult to recover an illumination distribution from image brightness observed on an object surface. The main reason for this difficulty is that there is usually not adequate variation in the image brightness observed on the object surface to reflect the subtle characteristics of the entire illumination. In this study, we demonstrate the effectiveness of using occluding information of incoming light in estimating an illumination distribution of a scene. Shadows in a real scene are caused by the occlusion of incoming light and, thus, analyzing the relationships between the image brightness and the occlusions of incoming light enables us to reliably estimate an illumination distribution of a scene even in a complex illumination environment. This study further concerns the following two issues that need to be addressed. First, the method combines the illumination analysis with an estimation of the reflectance properties of a shadow surface. This makes the method applicable to the case where reflectance properties of a surface are not known a priori and enlarges the variety of images applicable to the method. Second, we introduce an adaptive sampling framework for efficient estimation of illumination distribution. Using this framework, we are able to avoid a unnecessarily dense sampling of the illumination and can estimate the entire illumination distribution more efficiently with a smaller number of sampling directions of the illumination distribution. To demonstrate the effectiveness of the proposed method, we have successfully tested the proposed method by using sets of real images taken in natural illumination conditions with different surface materials of shadow regions.

Background subtraction based on cooccurrence of image variations

Abstract: This paper presents a novel background subtraction method for detecting foreground objects in dynamic scenes involving swaying trees and fluttering flags. Most methods proposed so far adjust the permissible range of the background image variations according to the training samples of background images. Thus, the detection sensitivity decreases at those pixels having wide permissible ranges. If we can narrow the ranges by analyzing input images, the detection sensitivity can be improved. For this narrowing, we employ the property that image variations at neighboring image blocks have strong correlation, also known as "cooccurrence". This approach is essentially different from chronological background image updating or morphological postprocessing. Experimental results for real images demonstrate the effectiveness of our method.

Eye gaze estimation from a single image of one eye

Abstract: We present a novel approach, called the "one-circle " algorithm, for measuring the eye gaze using a monocular image that zooms in on only one eye of a person. Observing that the iris contour is a circle, we estimate the normal direction of this iris circle, considered as the eye gaze, from its elliptical image. From basic projective geometry, an ellipse can be back-projected into space onto two circles of different orientations. However, by using an anthropometric property of the eyeball, the correct solution can be disambiguated. This allows us to obtain a higher resolution image of the iris with a zoom-in camera and thereby achieving higher accuracies in the estimation. The robustness of our gaze determination approach was verified statistically by the extensive experiments on synthetic and real image data. The two key contributions are that we show the possibility of finding the unique eye gaze direction from a single image of one eye and that one can obtain better accuracy as a consequence of this.

Segmentation of multiple salient closed contours from real images

Abstract: Using a saliency measure based on the global property of contour closure, we have developed a segmentation method which identifies smooth closed contours bounding objects of unknown shape in real images. The saliency measure incorporates the Gestalt principles of proximity and good continuity that previous methods have also exploited. Unlike previous methods, we incorporate contour closure by finding the eigenvector with the largest positive real eigenvalue of a transition matrix for a Markov process where edges from the image serve as states. Element (i, j) of the transition matrix is the conditional probability that a contour which contains edge j will also contain edge i. We show how the saliency measure, defined for individual edges, can be used to derive a saliency relation, defined for pairs of edges, and further show that strongly-connected components of the graph representing the saliency relation correspond to smooth closed contours in the image. Finally, we report for the first time, results on large real images for which segmentation takes an average of about 10 seconds per object on a general-purpose workstation.

Wavelet-based level set evolution for classification of textured images

Abstract: We present a supervised classification model based on a variational approach. This model is specifically devoted to textured images. We want to get a partition of an image, composed of texture regions separated by regular interfaces. Each kind of texture defines a class. We use a wavelet packet transform to analyze the textures, characterized by their energy distribution in each sub-band. In order to have an image segmentation according to the classes, we model the regions and their interfaces by level set functions. We define a functional on these level sets whose minimizers define the optimal classification according to texture. A system of coupled PDEs is deduced from the functional. By solving this system, each region evolves according to its wavelet coefficients and interacts with the neighbor regions in order to obtain a partition with regular contours. Experiments are shown on synthetic and real images.

Voxel carving for specular surfaces

Abstract: We present an novel algorithm that reconstructs voxels of a general 3D specular surface from multiple images of a calibrated camera. A calibrated scene (i.e. points whose 3D coordinates are known) is reflected by the unknown specular surface onto the image plane of the camera. For every viewpoint, surface normals are associated to the voxels traversed by each projection ray formed by the reflection of a scene point. A decision process then discards voxels whose associated surface normals are not consistent with one another. The output of the algorithm is a collection of voxels and surface normals in 3D space, whose quality and size depend on user-set thresholds. The method has been tested on synthetic and real images. Visual and quantified experimental results are presented.

Recovering 3-D shape and reflectance from a small number of photographs

Abstract: There are computer graphics applications for which the shape and reflectance of complex objects, such as faces, cannot be obtained using specialized equipment due to cost and practical considerations. We present an image based technique that uses only a small number of example images, and assumes a parametric model of reflectance, to simultaneously and reliably recover the Bidirectional Reflectance Distributions Function (BRDF) and the 3-D shape of non-Lambertian objects. No information about the position and intensity of the light-sources or the position of the camera is required. We successfully apply this approach to human faces, accurately recovering their 3-D shape and BRDF. We use the recovered information to efficiently and accurately render photorealistic images of the faces under novel illumination conditions in which the rendered image intensity closely matches the intensity in real images. The accuracy of our technique is further demonstrated by the close resemblance of the skin BRDF recovered using our method, to the one measured with a method presented in the literature and in which a 3-D scanner was used.

Motion recognition using nonparametric image motion models estimated from temporal and multiscale co-occurrence statistics

Abstract: A new approach for motion characterization in image sequences is presented. It relies on the probabilistic modeling of temporal and scale co-occurrence distributions of local motion-related measurements directly computed over image sequences. Temporal multiscale Gibbs models allow us to handle both spatial and temporal aspects of image motion content within a unified statistical framework. Since this modeling mainly involves the scalar product between co-occurrence values and Gibbs potentials, we can formulate and address several fundamental issues: model estimation according to the ML criterion (hence, model training and learning) and motion classification. We have conducted motion recognition experiments over a large set of real image sequences comprising various motion types such as temporal texture samples, human motion examples, and rigid motion situations.

Tracking objects using density matching and shape priors

Abstract: We present a novel method for tracking objects by combining density matching with shape priors. Density matching is a tracking method which operates by maximizing the Bhattacharyya similarity measure between the photometric distribution from an estimated image region and a model photometric distribution. Such trackers can be expressed as PDE-based curve evolutions, which can be implemented using level sets. Shape priors can be combined with this level-set implementation of density matching by representing the shape priors as a series of level sets; a variational approach allows for a natural, parametrization-independent shape term to be derived. Experimental results on real image sequences are shown.

Computer-generated holograms of three-dimensional objects synthesized from their multiple angular viewpoints

Abstract: Synthesizing computer-generated holograms (CGHs) of a general three-dimensional (3D) object is usually a heavy computational task. We propose and demonstrate a new algorithm for computing CGHs of 3D objects. In our scheme, many different angular projections of computer-designed 3D objects are numerically processed to yield a single two-dimensional complex matrix. This matrix is equivalent to the complex amplitude of a wave front on the rear focal plane of a spherical lens when the object is located near the front focal point and illuminated by a plane wave. Therefore the computed matrix can be used as a CGH after it is encoded to a real positive-valued transparency. When such CGH is illuminated by a plane wave, a 3D real image of the objects is constructed. The number of computer operations are equivalent to those of a two-dimensional Fourier CGH. Computer and optical constructions of 3D objects, both of which show the feasibility of the proposed approach, are described.

Constrained Structure and Motion From Multiple Uncalibrated Views of a Piecewise Planar Scene

Abstract: This paper is about multi-view modeling of a rigid scene. We merge the traditional approaches of reconstructing image-extractable features and of modeling via user-provided geometry. We use features to obtain a first guess for structure and motion, fit geometric primitives, correct the structure so that reconstructed features lie exactly on geometric primitives and optimize both structure and motion in a bundle adjustment manner while enforcing the underlying constraints. We specialize this general scheme to the point features and the plane geometric primitives. The underlying geometric relationships are described by multi-coplanarity constraints. We propose a minimal parameterization of the structure enforcing these constraints and use it to devise the corresponding maximum likelihood estimator. The recovered primitives are then textured from the input images. The result is an accurate and photorealistic model. Experimental results using simulated data confirm that the accuracy of the model using the constrained methods is of clearly superior quality compared to that of traditional methods and that our approach performs better than existing ones, for various scene configurations. In addition, we observe that the method still performs better in a number of configurations when the observed surfaces are not exactly planar. We also validate our method using real images.

Evaluation of local models of dynamic backgrounds

Abstract: Background subtraction is the first step of many video surveillance applications. What is considered background varies by application, and may include regular, systematic, or complex motions. The paper explores the use of several different local spatio-temporal models of a background, defined at each pixel in the image. We present experiments with real image data and conclude that appropriate local representations are sufficient to make background models of complicated real world motions. Empirical studies illustrate, for example, that an optical flow-based model is able to detect emergency vehicles whose motion is different from those typically observed in traffic scenes. We conclude that "different models are appropriate for different scenes", but give criteria by which one can choose which model will be best.

A perspective on distortions

Abstract: A framework for analyzing distortions in non-single viewpoint imaging systems is presented. Such systems possess loci of viewpoints called caustics. In general, perspective (or undistorted) views cannot be computed from images acquired with such systems without knowing scene structure. Views computed without scene structure will exhibit distortions, which we call caustic distortions. We first introduce a taxonomy of distortions based on the geometry of imaging systems. Then, we derive a metric to quantify caustic distortions. We present an algorithm to compute minimally distorted views using simple priors on scene structure. These priors are defined as parameterized primitives such as spheres, planes and cylinders with simple uncertainty models for the parameters. To validate our method, we conducted extensive experiments on rendered and real images. In all cases our method produces nearly undistorted views even though the acquired images were strongly distorted. We also provide an approximation of the above method that warps the entire captured image into a quasi single viewpoint representation that can be used by any "viewer" to compute near-perspective views in real-time.

Image display apparatus

Abstract: An in image display apparatus having a first display control unit for displaying a map image on a display panel on the basis of map data, a second display control unit for displaying real images of facilities such as ballparks, which images have the same scale of that of the map image, in areas corresponding to the facilities such as ballparks on the map image on the basis of area information indicating the areas corresponding to the facilities such as ballparks on the map image, and real image data associated with position coordinates.

Catadioptric camera calibration using geometric invariants

Abstract: Central catadioptric cameras are imaging devices that use mirrors to enhance the field of view while preserving a single effective viewpoint. In this paper, we propose a novel method for the calibration of central catadioptric cameras using geometric invariants. Lines in space are projected into conics in the catadioptric image plane as well as spheres in space. We proved that the projection of a line can provide three invariants whereas the projection of a sphere can provide two. From these invariants, constraint equations for the intrinsic parameters of catadioptric camera are derived. Therefore, there are two variants of this novel method. The first one uses the projections of lines and the second one uses the projections of spheres. In general, the projections of two lines or three spheres are sufficient to achieve the catadioptric camera calibration. One important observation in this paper is that the method based on the projections of spheres is more robust and has higher accuracy than that using the projections of lines. The performances of our method are demonstrated by the results of simulations and experiments with real images.

Short throw projection system and method

Abstract: A short throw projection system and method for displaying a corrected optical image on a projection screen based on input image data that includes an electronic correction unit, an image projector and a reflection assembly. The electronic correction unit receives the input image data and generates pre-distorted image data. The image projector receives the pre-distorted image data from the electronic correction unit and projects a pre-distorted optical image that corresponds to the pre-distorted image data or a pre-distorted image compensated by the projection optic distortion. The optical reflection assembly is positioned in the optical path of the pre-distorted optical image to project an optical image on the projection screen. The reflection assembly can consist of various combinations of curved and planar mirrors as desired. The electronic correction unit is encoded to pre-distort the geometry of the image represented by the image data such that when the pre-distorted optical image is projected through the image projector and reflected within the reflection assembly, the optical and geometric distortions associated with the image projector and the mirrors within the reflection assembly are eliminated in the displayed optical image.

Rapid shadow generation in real-world lighting environments

Abstract: We propose a new algorithm that uses consumer-level graphics hardware to render shadows cast by synthetic objects and a real lighting environment. This has immediate benefit for interactive Augmented Reality applications, where synthetic objects must be accurately merged with real images. We show how soft shadows cast by direct and indirect illumination sources may be generated and composited into a background image at interactive rates. We describe how the sources of light (and hence shadow) affecting each point in an image can be efficiently encoded using a hierarchical shaft-based subdivision of line-space. This subdivision is then used to determine the sources of light that are occluded by synthetic objects, and we show how the contributions from these sources may be removed from a background image using facilities available on modern graphics hardware. A trade-off may be made at run-time between shadow accuracy and rendering cost, converging towards a result that is subjectively similar to that obtained using ray-tracing based differential rendering algorithms. Examples of the proposed technique are given for a variety of different lighting environments, and the visual fidelity of images generated by our algorithm is compared to both real photographs and synthetic images generated using non-real-time techniques.

Unsupervised image segmentation using local homogeneity analysis

Abstract: In this paper, a novel method is presented for unsupervised image segmentation based on local homogeneity analysis. First, a criterion for homogeneity of a certain pattern is proposed. Applying the criterion to local windows in the original image results in the "H-image". The high and low values of the H-image correspond to possible region boundaries and region interiors respectively. Then, a region growing method is used to segment the image based on the H-image. Finally, visually similar regions are merged together to avoid over-segmentation. Experimental results on real images show the effectiveness and robustness of the method.

Autofocus system and microscope

Abstract: An autofocus system according to the present invention comprises: a light source; a focusing illumination optical system that forms an optical image generated with light from the light source on a target object through an objective lens; a focusing image forming optical system that receives through the objective lens reflected light generated as the optical image is reflected off the target object and forms a reflected image of the optical image; a photoelectric converter that is provided at an image forming position at which the reflected image is formed by the focusing image forming optical system to detect the reflected image; a signal output device that outputs a signal for controlling a focus actuator based upon a signal corresponding to the reflected image obtained at the photoelectric converter; and an image forming position adjustment device that adjusts an offset quantity between a focus position of the objective lens and an image forming position of the optical image by moving at least one of the image forming position of the optical image and the image forming position of the reflected image along an optical axis.

Vision-based Detection of Stair-cases

Abstract: Stair-cases are useful environmental landmarks for navigation in mobility aids for the partially sighted. In this paper, a texture detection method using Gabor Filters is proposed to detect distant stair-cases. When close enough, stair-cases are then detected by looking for groups of concurrent lines, where convex and concave edges are partitioned using intensity variation information. Stair-case pose is estimated by a homography search approach. Using an a priori stair-case model, search criteria and constraints are established to find its vertical rotation and slope. These algorithms have been applied to both synthetic and real images with promising results.