Showing papers on "Segmentation-based object categorization published in 2004"

Efficient Graph-Based Image Segmentation

Abstract: This paper addresses the problem of segmenting an image into regions. We define a predicate for measuring the evidence for a boundary between two regions using a graph-based representation of the image. We then develop an efficient segmentation algorithm based on this predicate, and show that although this algorithm makes greedy decisions it produces segmentations that satisfy global properties. We apply the algorithm to image segmentation using two different kinds of local neighborhoods in constructing the graph, and illustrate the results with both real and synthetic images. The algorithm runs in time nearly linear in the number of graph edges and is also fast in practice. An important characteristic of the method is its ability to preserve detail in low-variability image regions while ignoring detail in high-variability regions.

Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation

Abstract: Characterizing the performance of image segmentation approaches has been a persistent challenge. Performance analysis is important since segmentation algorithms often have limited accuracy and precision. Interactive drawing of the desired segmentation by human raters has often been the only acceptable approach, and yet suffers from intra-rater and inter-rater variability. Automated algorithms have been sought in order to remove the variability introduced by raters, but such algorithms must be assessed to ensure they are suitable for the task. The performance of raters (human or algorithmic) generating segmentations of medical images has been difficult to quantify because of the difficulty of obtaining or estimating a known true segmentation for clinical data. Although physical and digital phantoms can be constructed for which ground truth is known or readily estimated, such phantoms do not fully reflect clinical images due to the difficulty of constructing phantoms which reproduce the full range of imaging characteristics and normal and pathological anatomical variability observed in clinical data. Comparison to a collection of segmentations by raters is an attractive alternative since it can be carried out directly on the relevant clinical imaging data. However, the most appropriate measure or set of measures with which to compare such segmentations has not been clarified and several measures are used in practice. We present here an expectation-maximization algorithm for simultaneous truth and performance level estimation (STAPLE). The algorithm considers a collection of segmentations and computes a probabilistic estimate of the true segmentation and a measure of the performance level represented by each segmentation. The source of each segmentation in the collection may be an appropriately trained human rater or raters, or may be an automated segmentation algorithm. The probabilistic estimate of the true segmentation is formed by estimating an optimal combination of the segmentations, weighting each segmentation depending upon the estimated performance level, and incorporating a prior model for the spatial distribution of structures being segmented as well as spatial homogeneity constraints. STAPLE is straightforward to apply to clinical imaging data, it readily enables assessment of the performance of an automated image segmentation algorithm, and enables direct comparison of human rater and algorithm performance.

Combined Object Categorization and Segmentation With an Implicit Shape Model

Abstract: We present a method for object categorization in real-world scenes. Following a common consensus in the field, we do not assume that a figure- ground segmentation is available prior to recognition. However, in contrast to most standard approaches for object class recognition, our approach automati- cally segments the object as a result of the categorization. This combination of recognition and segmentation into one process is made pos- sible by our use of an Implicit Shape Model, which integrates both into a common probabilistic framework. In addition to the recognition and segmentation result, it also generates a per-pixel confidence measure specifying the area that supports a hypothesis and how much it can be trusted. We use this confidence to derive a nat- ural extension of the approach to handle multiple objects in a scene and resolve ambiguities between overlapping hypotheses with a novel MDL-based criterion. In addition, we present an extensive evaluation of our method on a standard dataset for car detection and compare its performance to existing methods from the literature. Our results show that the proposed method significantly outper- forms previously published methods while needing one order of magnitude less training examples. Finally, we present results for articulated objects, which show that the proposed method can categorize and segment unfamiliar objects in differ- ent articulations and with widely varying texture patterns, even under significant partial occlusion.

Statistical region merging

Abstract: This paper explores a statistical basis for a process often described in computer vision: image segmentation by region merging following a particular order in the choice of regions. We exhibit a particular blend of algorithmics and statistics whose segmentation error is, as we show, limited from both the qualitative and quantitative standpoints. This approach can be efficiently approximated in linear time/space, leading to a fast segmentation algorithm tailored to processing images described using most common numerical pixel attribute spaces. The conceptual simplicity of the approach makes it simple to modify and cope with hard noise corruption, handle occlusion, authorize the control of the segmentation scale, and process unconventional data such as spherical images. Experiments on gray-level and color images, obtained with a short readily available C-code, display the quality of the segmentations obtained.

Improved watershed transform for medical image segmentation using prior information

Abstract: The watershed transform has interesting properties that make it useful for many different image segmentation applications: it is simple and intuitive, can be parallelized, and always produces a complete division of the image. However, when applied to medical image analysis, it has important drawbacks (oversegmentation, sensitivity to noise, poor detection of thin or low signal to noise ratio structures). We present an improvement to the watershed transform that enables the introduction of prior information in its calculation. We propose to introduce this information via the use of a previous probability calculation. Furthermore, we introduce a method to combine the watershed transform and atlas registration, through the use of markers. We have applied our new algorithm to two challenging applications: knee cartilage and gray matter/white matter segmentation in MR images. Numerical validation of the results is provided, demonstrating the strength of the algorithm for medical image segmentation.

Interactive Image Segmentation Using an Adaptive GMMRF Model

Abstract: The problem of interactive foreground/background segmentation in still images is of great practical importance in image editing. The state of the art in interactive segmentation is probably represented by the graph cut algorithm of Boykov and Jolly (ICCV 2001). Its underlying model uses both colour and contrast information, together with a strong prior for region coherence. Estimation is performed by solving a graph cut problem for which very efficient algorithms have recently been developed. However the model depends on parameters which must be set by hand and the aim of this work is for those constants to be learned from image data.

Recovering human body configurations: combining segmentation and recognition

Abstract: The goal of this work is to detect a human figure image and localize his joints and limbs along with their associated pixel masks. In this work we attempt to tackle this problem in a general setting. The dataset we use is a collection of sports news photographs of baseball players, varying dramatically in pose and clothing. The approach that we take is to use segmentation to guide our recognition algorithm to salient bits of the image. We use this segmentation approach to build limb and torso detectors, the outputs of which are assembled into human figures. We present quantitative results on torso localization, in addition to shortlisted full body configurations.

Chinese segmentation and new word detection using conditional random fields

Abstract: Chinese word segmentation is a difficult, important and widely-studied sequence modeling problem. This paper demonstrates the ability of linear-chain conditional random fields (CRFs) to perform robust and accurate Chinese word segmentation by providing a principled framework that easily supports the integration of domain knowledge in the form of multiple lexicons of characters and words. We also present a probabilistic new word detection method, which further improves performance. Our system is evaluated on four datasets used in a recent comprehensive Chinese word segmentation competition. State-of-the-art performance is obtained.

Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections.

Abstract: We present a region-based segmentation method in which seeds representing both object and background pixels are created by combining morphological filtering of both the original image and the gradient magnitude of the image. The seeds are then used as starting points for watershed segmentation of the gradient magnitude image. The fully automatic seeding is done in a generous fashion, so that at least one seed will be set in each foreground object. If more than one seed is placed in a single object, the watershed segmentation will lead to an initial over-segmentation, i.e. a boundary is created where there is no strong edge. Thus, the result of the initial segmentation is further refined by merging based on the gradient magnitude along the boundary separating neighbouring objects. This step also makes it easy to remove objects with poor contrast. As a final step, clusters of nuclei are separated, based on the shape of the cluster. The number of input parameters to the full segmentation procedure is only five. These parameters can be set manually using a test image and thereafter be used on a large number of images created under similar imaging conditions. This automated system was verified by comparison with manual counts from the same image fields. About 90% correct segmentation was achieved for two- as well as three-dimensional images.

Combining Top-Down and Bottom-Up Segmentation

Abstract: In this work we show how to combine bottom-up and top-down approaches into a single figure-ground segmentation process. This process provides accurate delineation of object boundaries that cannot be achieved by either the top-down or bottom-up approach alone. The top-down approach uses object representation learned from examples to detect an object in a given input image and provide an approximation to its figure-ground segmentation. The bottom-up approach uses image-based criteria to define coherent groups of pixels that are likely to belong together to either the figure or the background part. The combination provides a final segmentation that draws on the relative merits of both approaches: The result is as close as possible to the top-down approximation, but is also constrained by the bottom-up process to be consistent with significant image discontinuities. We construct a global cost function that represents these top-down and bottom-up requirements. We then show how the global minimum of this function can be efficiently found by applying the sum-product algorithm. This algorithm also provides a confidence map that can be used to identify image regions where additional top-down or bottom-up information may further improve the segmentation. Our experiments show that the results derived from the algorithm are superior to results given by a pure top-down or pure bottom-up approach. The scheme has broad applicability, enabling the combined use of a range of existing bottom-up and top-down segmentations.

Skin color-based video segmentation under time-varying illumination

Abstract: A novel approach for real-time skin segmentation in video sequences is described. The approach enables reliable skin segmentation despite wide variation in illumination during tracking. An explicit second order Markov model is used to predict evolution of the skin-color (HSV) histogram over time. Histograms are dynamically updated based on feedback from the current segmentation and predictions of the Markov model. The evolution of the skin-color distribution at each frame is parameterized by translation, scaling, and rotation in color space. Consequent changes in geometric parameterization of the distribution are propagated by warping and resampling the histogram. The parameters of the discrete-time dynamic Markov model are estimated using maximum likelihood estimation and also evolve over time. The accuracy of the new dynamic skin color segmentation algorithm is compared to that obtained via a static color model. Segmentation accuracy is evaluated using labeled ground-truth video sequences taken from staged experiments and popular movies. An overall increase in segmentation accuracy of up to 24 percent is observed in 17 out of 21 test sequences. In all but one case, the skin-color classification rates for our system were higher, with background classification rates comparable to those of the static segmentation.

Segmentation of 3D meshes through spectral clustering

Abstract: We formulate and apply spectral clustering to 3D mesh segmentation for the first time and report our preliminary findings Given a set of mesh faces, an affinity matrix which encodes the likelihood of each pair of faces belonging to the same group is first constructed Spectral methods then use selected eigenvectors of the affinity matrix or its closely related graph Laplacian to obtain data representations that can be more easily clustered We develop an algorithm that favors segmentation along concave regions, which is inspired by human perception Our algorithm is theoretically sound, efficient, simple to implement, andean achieve high-quality segmentation results on 3D meshes

Image and Video Segmentation by Anisotropic Kernel Mean Shift

Abstract: Mean shift is a nonparametric estimator of density which has been applied to image and video segmentation. Traditional mean shift based segmentation uses a radially symmetric kernel to estimate local density, which is not optimal in view of the often structured nature of image and more particularly video data. In this paper we present an anisotropic kernel mean shift in which the shape, scale, and orientation of the kernels adapt to the local structure of the image or video. We decompose the anisotropic kernel to provide handles for modifying the segmentation based on simple heuristics. Experimental results show that the anisotropic kernel mean shift outperforms the original mean shift on image and video segmentation in the following aspects: 1) it gets better results on general images and video in a smoothness sense; 2) the segmented results are more consistent with human visual saliency; 3) the algorithm is robust to initial parameters.

Image Segmentation Methods for Object-based Analysis and Classification

Abstract: Th e continuously improving spatial resolution of remote sensing (RS) sensors sets new demand for applications utilising this information. Th e need for the more effi cient extraction of information from high resolution RS imagery and the seamless integration of this information into Geographic Information System (GIS) databases is driving geo-information theory and methodology into new territory. As the dimension of the ground instantaneous fi eld of view (GIFOV), or pixel (picture element) size, decreases many more fi ne landscape features can be readily delineated, at least visually. Th e challenge has been to produce proven man-machine methods that externalize and improve on human interpretation skills. Some of the most promising results in this research programme have come from the adoption of image segmentation algorithms and the development of so-called object-based classifi cation methodologies. In this chapter we describe diff erent approaches to image segmentation and explore how segmentation and object-based methods improve on traditional pixel-based image analysis/classifi cation methods. According to Schowengerdt () the traditional image processing/image classifi cation methodology is referred to as an image-centred approach. Here, the primary goal is to produce a map describing the spatial relationships between phenomena of interest. A second type, the data-centred approach, is pursued when the user is primarily interested in estimating parameters for individual phenomena based on the data values. Due to recent developments in image processing the two approaches appear to be converging: from image and data centred views to an information-centred approach. For instance, for change detection and environmental monitoring tasks we must not only extract information from the spectral and temporal data dimensions. We must also integrate these estimates into a spatial framework and make a priori and a posteriori utilization of GIS databases. A decision support system must encapsulate manager knowledge, context/ecological knowledge and planning knowledge. Technically, this necessitates a closer integration of remote sensing and GIS methods. Ontologically, it demands a new methodology that can provide a fl exible, demanddriven generation of information and, consequently, hierarchically structured semantic rules describing the relationships between the diff erent levels of spatial entities. Several of the aspects of geo-information involved cannot be obtained by pixel information as such but can only be achieved with an exploitation of neighbourhood information and context of the objects of interest. Th e relationship between ground objects and image objects

Multi-label Image Segmentation for Medical Applications Based on Graph-Theoretic Electrical Potentials

Abstract: A novel method is proposed for performing multi-label, semi-automated image segmentation. Given a small number of pixels with user-defined labels, one can analytically (and quickly) determine the probability that a random walker starting at each unlabeled pixel will first reach one of the pre-labeled pixels. By assigning each pixel to the label for which the greatest probability is calculated, a high-quality image segmentation may be obtained. Theoretical properties of this algorithm are developed along with the corresponding connections to discrete potential theory and electrical circuits. This algorithm is formulated in discrete space (i.e., on a graph) using combinatorial analogues of standard operators and principles from continuous potential theory, allowing it to be applied in arbitrary dimension.

Normalized cuts in 3-D for spinal MRI segmentation

Abstract: Segmentation of medical images has become an indispensable process to perform quantitative analysis of images of human organs and their functions. Normalized Cuts (NCut) is a spectral graph theoretic method that readily admits combinations of different features for image segmentation. The computational demand imposed by NCut has been successfully alleviated with the Nystro/spl uml/m approximation method for applications different than medical imaging. In this paper we discuss the application of NCut with the Nystro/spl uml/m approximation method to segment vertebral bodies from sagittal T1-weighted magnetic resonance images of the spine. The magnetic resonance images were preprocessed by the anisotropic diffusion algorithm, and three-dimensional local histograms of brightness was chosen as the segmentation feature. Results of the segmentation as well as limitations and challenges in this area are presented.

Automated image segmentation using improved PCNN model based on cross-entropy

Abstract: The pulse coupled neural network (PCNN) is a new neural network that was developed and formed in the 1990's. The key point of a PCNN is the modulated coupling mechanism, while coupled results produce internal activity. The output of the PCNN is a binary image sequence, which can be considered the result of threshold segmentation. In this paper, the matrix made by the internal activity is regarded as a breadth of image, which then can be conjoined with the technique of traditional threshold segmentation. The application of the minimum cross-entropy criterion in the technique of image segmentation makes the discrepancy of information content between segmented image and image after segmentation to be minimal. A kind of novel of image segmentation algorithm based on automatic cycle iterations is put forward, after the traditional PCNN threshold segmentation mechanism is improved in combination with the minimum cross-entropy criterion. Theory analysis and experimental results all show that the best segmentation output can be drawn using this new algorithm.

Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval

Abstract: In this paper, a novel algorithm is presented for the real-time, compressed-domain, unsupervised segmentation of image sequences and is applied to video indexing and retrieval. The segmentation algorithm uses motion and color information directly extracted from the MPEG-2 compressed stream. An iterative rejection scheme based on the bilinear motion model is used to effect foreground/background segmentation. Following that, meaningful foreground spatiotemporal objects are formed by initially examining the temporal consistency of the output of iterative rejection, clustering the resulting foreground macroblocks to connected regions and finally performing region tracking. Background segmentation to spatiotemporal objects is additionally performed. MPEG-7 compliant low-level descriptors describing the color, shape, position, and motion of the resulting spatiotemporal objects are extracted and are automatically mapped to appropriate intermediate-level descriptors forming a simple vocabulary termed object ontology. This, combined with a relevance feedback mechanism, allows the qualitative definition of the high-level concepts the user queries for (semantic objects, each represented by a keyword) and the retrieval of relevant video segments. Desired spatial and temporal relationships between the objects in multiple-keyword queries can also be expressed, using the shot ontology. Experimental results of the application of the segmentation algorithm to known sequences demonstrate the efficiency of the proposed segmentation approach. Sample queries reveal the potential of employing this segmentation algorithm as part of an object-based video indexing and retrieval scheme.

Segmentation of the face and hands in sign language video sequences using color and motion cues

Abstract: We present a hand and face segmentation methodology using color and motion cues for the content-based representation of sign language video sequences. The methodology consists of three stages: skin-color segmentation; change detection; face and hand segmentation mask generation. In skin-color segmentation, a universal color-model is derived and image pixels are classified as skin or nonskin based on their Mahalanobis distance. We derive a segmentation threshold for the classifier. The aim of change detection is to localize moving objects in a video sequences. The change detection technique is based on the F test and block-based motion estimation. Finally, the results from skin-color segmentation and change detection are analyzed to segment the face and hands. The performance of the algorithm is illustrated by simulations carried out on standard test sequences.

Performance measures for video object segmentation and tracking

Abstract: We propose measures to evaluate quantitatively the performance of video object segmentation and tracking methods without ground-truth (GT) segmentation maps. The proposed measures are based on spatial differences of color and motion along the boundary of the estimated video object plane and temporal differences between the color histogram of the current object plane and its predecessors. They can be used to localize (spatially and/or temporally) regions where segmentation results are good or bad; and/or they can be combined to yield a single numerical measure to indicate the goodness of the boundary segmentation and tracking results over a sequence. The validity of the proposed performance measures without GT have been demonstrated by canonical correlation analysis with another set of measures with GT on a set of sequences (where GT information is available). Experimental results are presented to evaluate the segmentation maps obtained from various sequences using different segmentation approaches.

A Second Order Shape Optimization Approach for Image Segmentation

Abstract: The problem of segmentation of a given image using the active contour technique is considered. An abstract calculus to find appropriate speed functions for active contour models in image segmentation or related problems based on variational principles is presented. The speed method from shape sensitivity analysis is used to derive speed functions which correspond to gradient or Newton-type directions for the underlying optimization problem. The Newton-type speed function is found by solving an elliptic problem on the current active contour in every time step. Numerical experiments comparing the classical gradient method with Newton's method are presented.

Fast video segmentation algorithm with shadow cancellation, global motion compensation, and adaptive threshold techniques

Abstract: Automatic video segmentation plays an important role in real-time MPEG-4 encoding systems. Several video segmentation algorithms have been proposed; however, most of them are not suitable for real-time applications because of high computation load and many parameters needed to be set in advance. This paper presents a fast video segmentation algorithm for MPEG-4 camera systems. With change detection and background registration techniques, this algorithm can give satisfying segmentation results with low computation load. The processing speed of 40 QCIF frames per second can be achieved on a personal computer with an 800 MHz Pentium-III processor. Besides, it has shadow cancellation mode, which can deal with light changing effect and shadow effect. A fast global motion compensation algorithm is also included in this algorithm to make it applicable in slight moving camera situations. Furthermore, the required parameters can be decided automatically, which can enhance the proposed algorithm to have adaptive threshold ability. It can be integrated into MPEG-4 videophone systems and digital cameras.

Unsupervised image segmentation using a simple MRF model with a new implementation scheme

Abstract: A Markov random field (MRF) model with a new implementation scheme is proposed for unsupervised image segmentation based on image features. The traditional two-component MRF model for segmentation requires training data to estimate necessary model parameters and is thus unsuitable for unsupervised segmentation. The new MRF model overcomes this problem by introducing a function-based weighting parameter between the two components. This new MRF model is able to automatically estimate model parameters and is demonstrated to produce more accurate image segmentations than the traditional model using a variety of imagery.

Segmentation of the Liver using a 3D Statistical Shape Model

Abstract: This paper presents an automatic approach for segmentation of the liver from computer tomography (CT) images based on a 3D statistical shape model. Segmentation of the liver is an important prerequisite in liver surgery planning. One of the major challenges in building a 3D shape model from a training set of segmented instances of an object is the determination of the correspondence between different surfaces. We propose to use a geometric approach that is based on minimizing the distortion of the correspondence mapping between two different surfaces. For the adaption of the shape model to the image data a profile model based on the grey value appearance of the liver and its surrounding tissues in contrast enhanced CT data was developed. The robustness of this method results from a previous nonlinear diffusion filtering of the image data. Special focus is turned to the quantitative evaluation of the segmentation process. Several different error measures are discussed and implemented in a study involving more than 30 livers.

Interactive volume segmentation with differential image foresting transforms

Abstract: The absence of object information very often asks for considerable human assistance in medical image segmentation. Many interactive two-dimensional and three-dimensional (3-D) segmentation methods have been proposed, but their response time to user's actions should be considerably reduced to make them viable from the practical point of view. We circumvent this problem in the framework of the image foresting transform (IFT)-a general tool for the design of image operators based on connectivity-by introducing a new algorithm (DIFT) to compute sequences of IFTs in a differential way. We instantiate the DIFT algorithm for watershed-based and fuzzy-connected segmentations under two paradigms (single-object and multiple-object) and evaluate the efficiency gains of both approaches with respect to their linear-time implementation based on the nondifferential IFT. We show that the DIFT algorithm provides efficiency gains from 10 to 17, reducing the user's waiting time for segmentation with 3-D visualization on a common PC from 19-36 s to 2-3 s. We also show that the multiple-object approach is more efficient than the single-object paradigm for both segmentation methods.

A 3D statistical shape model of the pelvic bone for segmentation

Abstract: Statistical models of shape are a promising approach for robust and automatic segmentation of medical image data. This work describes the construction of a statistical shape model of the pelvic bone. An interactive approach is proposed for solving the correspondence problem which is able to handle shapes of arbitrary topology, suitable for the genus 3 surface of the pelvic bone. Moreover it allows to specify corresponding anatomical features as boundary constraints to the matching process. The model's capability for segmentation was tested on a set of 23 CT data sets. Quantitative results will be presented, showing that the model is well suited for segmentation purposes.