scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Efficient Graph-Based Image Segmentation

01 Sep 2004-International Journal of Computer Vision (Kluwer Academic Publishers)-Vol. 59, Iss: 2, pp 167-181
TL;DR: An efficient segmentation algorithm is developed based on a predicate for measuring the evidence for a boundary between two regions using a graph-based representation of the image and it is shown that although this algorithm makes greedy decisions it produces segmentations that satisfy global properties.
Abstract: This paper addresses the problem of segmenting an image into regions. We define a predicate for measuring the evidence for a boundary between two regions using a graph-based representation of the image. We then develop an efficient segmentation algorithm based on this predicate, and show that although this algorithm makes greedy decisions it produces segmentations that satisfy global properties. We apply the algorithm to image segmentation using two different kinds of local neighborhoods in constructing the graph, and illustrate the results with both real and synthetic images. The algorithm runs in time nearly linear in the number of graph edges and is also fast in practice. An important characteristic of the method is its ability to preserve detail in low-variability image regions while ignoring detail in high-variability regions.

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI
TL;DR: A new superpixel algorithm is introduced, simple linear iterative clustering (SLIC), which adapts a k-means clustering approach to efficiently generate superpixels and is faster and more memory efficient, improves segmentation performance, and is straightforward to extend to supervoxel generation.
Abstract: Computer vision applications have come to rely increasingly on superpixels in recent years, but it is not always clear what constitutes a good superpixel algorithm. In an effort to understand the benefits and drawbacks of existing methods, we empirically compare five state-of-the-art superpixel algorithms for their ability to adhere to image boundaries, speed, memory efficiency, and their impact on segmentation performance. We then introduce a new superpixel algorithm, simple linear iterative clustering (SLIC), which adapts a k-means clustering approach to efficiently generate superpixels. Despite its simplicity, SLIC adheres to boundaries as well as or better than previous methods. At the same time, it is faster and more memory efficient, improves segmentation performance, and is straightforward to extend to supervoxel generation.

7,849 citations


Cites background or methods from "Efficient Graph-Based Image Segment..."

  • ...Felzenszwalb and Huttenlocher [8] propose an alter- native graph-based approach that has been applied to generate superpixels....

    [...]

  • ...We therefore performed an empirical comparison of five stateof-the-art superpixel methods [8], [23], [26], [25], [15], evaluating their speed, ability to adhere to image boundaries, and impact on segmentation performance....

    [...]

  • ...Felzenszwalb and Huttenlocher [8] propose an alter-...

    [...]

  • ...Like some other superpixel algorithms [8], SLIC does not explicitly...

    [...]

  • ...For example, if adherence to image boundaries is of paramount importance, the graph-based method of [8] may be an ideal choice....

    [...]

Journal ArticleDOI
TL;DR: This paper introduces selective search which combines the strength of both an exhaustive search and segmentation, and shows that its selective search enables the use of the powerful Bag-of-Words model for recognition.
Abstract: This paper addresses the problem of generating possible object locations for use in object recognition. We introduce selective search which combines the strength of both an exhaustive search and segmentation. Like segmentation, we use the image structure to guide our sampling process. Like exhaustive search, we aim to capture all possible object locations. Instead of a single technique to generate possible object locations, we diversify our search and use a variety of complementary image partitionings to deal with as many image conditions as possible. Our selective search results in a small set of data-driven, class-independent, high quality locations, yielding 99 % recall and a Mean Average Best Overlap of 0.879 at 10,097 locations. The reduced number of locations compared to an exhaustive search enables the use of stronger machine learning techniques and stronger appearance models for object recognition. In this paper we show that our selective search enables the use of the powerful Bag-of-Words model for recognition. The selective search software is made publicly available (Software: http://disi.unitn.it/~uijlings/SelectiveSearch.html ).

5,843 citations

01 Oct 2008
TL;DR: The database contains labeled face photographs spanning the range of conditions typically encountered in everyday life, and exhibits “natural” variability in factors such as pose, lighting, race, accessories, occlusions, and background.
Abstract: Most face databases have been created under controlled conditions to facilitate the study of specific parameters on the face recognition problem. These parameters include such variables as position, pose, lighting, background, camera quality, and gender. While there are many applications for face recognition technology in which one can control the parameters of image acquisition, there are also many applications in which the practitioner has little or no control over such parameters. This database, Labeled Faces in the Wild, is provided as an aid in studying the latter, unconstrained, recognition problem. The database contains labeled face photographs spanning the range of conditions typically encountered in everyday life. The database exhibits “natural” variability in factors such as pose, lighting, race, accessories, occlusions, and background. In addition to describing the details of the database, we provide specific experimental paradigms for which the database is suitable. This is done in an effort to make research performed with the database as consistent and comparable as possible. We provide baseline results, including results of a state of the art face recognition system combined with a face alignment system. To facilitate experimentation on the database, we provide several parallel databases, including an aligned version.

5,742 citations


Cites methods from "Efficient Graph-Based Image Segment..."

  • ...We also experimented with the Felzenszwalb and Huttenlocher [24] algorithm but found that Mori’s method, while more computationally expensive, did a much better job at preserving the face-background boundary, a crucial property for superpixel-based segmentation....

    [...]

Journal ArticleDOI
TL;DR: This paper investigates two fundamental problems in computer vision: contour detection and image segmentation and presents state-of-the-art algorithms for both of these tasks.
Abstract: This paper investigates two fundamental problems in computer vision: contour detection and image segmentation. We present state-of-the-art algorithms for both of these tasks. Our contour detector combines multiple local cues into a globalization framework based on spectral clustering. Our segmentation algorithm consists of generic machinery for transforming the output of any contour detector into a hierarchical region tree. In this manner, we reduce the problem of image segmentation to that of contour detection. Extensive experimental evaluation demonstrates that both our contour detection and segmentation methods significantly outperform competing algorithms. The automatically generated hierarchical segmentations can be interactively refined by user-specified annotations. Computation at multiple image resolutions provides a means of coupling our system to recognition applications.

5,068 citations

Proceedings ArticleDOI
20 Jun 2011
TL;DR: This work proposes a regional contrast based saliency extraction algorithm, which simultaneously evaluates global contrast differences and spatial coherence, and consistently outperformed existing saliency detection methods.
Abstract: Automatic estimation of salient object regions across images, without any prior assumption or knowledge of the contents of the corresponding scenes, enhances many computer vision and computer graphics applications. We introduce a regional contrast based salient object detection algorithm, which simultaneously evaluates global contrast differences and spatial weighted coherence scores. The proposed algorithm is simple, efficient, naturally multi-scale, and produces full-resolution, high-quality saliency maps. These saliency maps are further used to initialize a novel iterative version of GrabCut, namely SaliencyCut, for high quality unsupervised salient object segmentation. We extensively evaluated our algorithm using traditional salient object detection datasets, as well as a more challenging Internet image dataset. Our experimental results demonstrate that our algorithm consistently outperforms 15 existing salient object detection and segmentation methods, yielding higher precision and better recall rates. We also show that our algorithm can be used to efficiently extract salient object masks from Internet images, enabling effective sketch-based image retrieval (SBIR) via simple shape comparisons. Despite such noisy internet images, where the saliency regions are ambiguous, our saliency guided image retrieval achieves a superior retrieval rate compared with state-of-the-art SBIR methods, and additionally provides important target object region information.

3,653 citations


Cites methods from "Efficient Graph-Based Image Segment..."

  • ...We first segment the input image into regions using a graphbased image segmentation method [11]....

    [...]

  • ...Image regions generated by Felzenszwalb and Huttenlocher’s segmentation method [11] (left), region contrast based segmentation with (left-middle) and without (right-middle) distance weighting....

    [...]

  • ...In contrast, our RC variant is slower as it requires image segmentation [11], but produces superior quality saliency maps....

    [...]

References
More filters
Book
01 Jan 1990
TL;DR: The updated new edition of the classic Introduction to Algorithms is intended primarily for use in undergraduate or graduate courses in algorithms or data structures and presents a rich variety of algorithms and covers them in considerable depth while making their design and analysis accessible to all levels of readers.
Abstract: From the Publisher: The updated new edition of the classic Introduction to Algorithms is intended primarily for use in undergraduate or graduate courses in algorithms or data structures. Like the first edition,this text can also be used for self-study by technical professionals since it discusses engineering issues in algorithm design as well as the mathematical aspects. In its new edition,Introduction to Algorithms continues to provide a comprehensive introduction to the modern study of algorithms. The revision has been updated to reflect changes in the years since the book's original publication. New chapters on the role of algorithms in computing and on probabilistic analysis and randomized algorithms have been included. Sections throughout the book have been rewritten for increased clarity,and material has been added wherever a fuller explanation has seemed useful or new information warrants expanded coverage. As in the classic first edition,this new edition of Introduction to Algorithms presents a rich variety of algorithms and covers them in considerable depth while making their design and analysis accessible to all levels of readers. Further,the algorithms are presented in pseudocode to make the book easily accessible to students from all programming language backgrounds. Each chapter presents an algorithm,a design technique,an application area,or a related topic. The chapters are not dependent on one another,so the instructor can organize his or her use of the book in the way that best suits the course's needs. Additionally,the new edition offers a 25% increase over the first edition in the number of problems,giving the book 155 problems and over 900 exercises thatreinforcethe concepts the students are learning.

21,651 citations

Proceedings ArticleDOI
17 Jun 1997
TL;DR: This work treats image segmentation as a graph partitioning problem and proposes a novel global criterion, the normalized cut, for segmenting the graph, which measures both the total dissimilarity between the different groups as well as the total similarity within the groups.
Abstract: We propose a novel approach for solving the perceptual grouping problem in vision. Rather than focusing on local features and their consistencies in the image data, our approach aims at extracting the global impression of an image. We treat image segmentation as a graph partitioning problem and propose a novel global criterion, the normalized cut, for segmenting the graph. The normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups. We show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion. We have applied this approach to segmenting static images and found results very encouraging.

11,827 citations


"Efficient Graph-Based Image Segment..." refers background or methods in this paper

  • ...The work reported here and the normalized cuts approach (Shi and Malik, 1997) are just a few illustrations of these recent advances....

    [...]

  • ...This bias was addressed with the normalized cut criterion developed by Shi and Malik (1997), which takes into account self-similarity of regions....

    [...]

  • ...…techniques (e.g., Cooper, 1998; Pavlidas, 1977), techniques based on mapping image pixels to some feature space (e.g., Comaniciu and Meer, 1997, 1999) and more recent formulations in terms of graph cuts (e.g., Shi and Malik, 1997; Wu and Leahy, 1993) and spectral methods (e.g., Weiss, 1999)....

    [...]

  • ...3 shows two baseball players (from Shi and Malik, 1997)....

    [...]

  • ...While the past few years have seen considerable progress in eigenvector-based methods of image segmentation (e.g., Shi and Malik, 1997; Weiss, 1999), these methods are too slow to be practical for many applications....

    [...]

01 Jan 1988

9,439 citations


"Efficient Graph-Based Image Segment..." refers background in this paper

  • ...There is a large literature on segmentation and clustering, dating back over 30 years, with applications in many areas other than computer vision (cf. Jain and Dubes, 1988)....

    [...]

  • ...One common approach to image segmentation is based on mapping each pixel to a point in some feature space, and then finding clusters of similar points (e.g., Comaniciu and Meer, 1997, 1999; Jain and Dubes, 1988)....

    [...]

  • ...A number of approaches to segmentation are based on finding compact clusters in some feature space (cf. Comaniciu and Meer, 1997; Jain and Dubes, 1988)....

    [...]

Book
01 Jan 1988

8,586 citations

Journal ArticleDOI
TL;DR: A family of graph-theoretical algorithms based on the minimal spanning tree are capable of detecting several kinds of cluster structure in arbitrary point sets; description of the detected clusters is possible in some cases by extensions of the method.
Abstract: A family of graph-theoretical algorithms based on the minimal spanning tree are capable of detecting several kinds of cluster structure in arbitrary point sets; description of the detected clusters is possible in some cases by extensions of the method. Development of these clustering algorithms was based on examples from two-dimensional space because we wanted to copy the human perception of gestalts or point groupings. On the other hand, all the methods considered apply to higher dimensional spaces and even to general metric spaces. Advantages of these methods include determinacy, easy interpretation of the resulting clusters, conformity to gestalt principles of perceptual organization, and invariance of results under monotone transformations of interpoint distance. Brief discussion is made of the application of cluster detection to taxonomy and the selection of good feature spaces for pattern recognition. Detailed analyses of several planar cluster detection problems are illustrated by text and figures. The well-known Fisher iris data, in four-dimensional space, have been analyzed by these methods also. PL/1 programs to implement the minimal spanning tree methods have been fully debugged.

1,832 citations


"Efficient Graph-Based Image Segment..." refers methods in this paper

  • ...In this section we briefly consider some of the related work that is most relevant to our approach: early graph-based methods (e.g., Urquhart, 1982; Zahn, 1971), region merging techniques (e.g., Cooper, 1998; Pavlidas, 1977), techniques based on mapping image pixels to some feature space (e.g.,…...

    [...]

  • ...The work of Zahn (1971) presents a segmentation method based on the minimum spanning tree (MST) of the graph....

    [...]

  • ...In this section we briefly consider some of the related work that is most relevant to our approach: early graph-based methods (e.g., Urquhart, 1982; Zahn, 1971), region merging techniques (e....

    [...]

  • ...As with certain classical clustering methods (Urquhart, 1982; Zahn, 1971), our method is based on selecting edges from a graph, where each pixel corresponds to a node in the graph, and certain neighboring pixels are connected by undirected edges....

    [...]

  • ...As in other graph-based approaches to image segmentation (e.g., Shi and Malik, 1997; Wu and Leahy, 1993; Zahn, 1971) we define an undirected graph G = (V, E), where each image pixel pi has a corresponding vertex vi ∈ V ....

    [...]