Graph-Based Visual Saliency

Home
/
Papers
/
Graph-Based Visual Saliency

Proceedings Article•

Graph-Based Visual Saliency

Jonathan Harel¹, Christof Koch¹, Pietro Perona¹•Institutions (1)

04 Dec 2006-Vol. 19, pp 545-552

TL;DR: A new bottom-up visual saliency model, Graph-Based Visual Saliency (GBVS), is proposed, which powerfully predicts human fixations on 749 variations of 108 natural images, achieving 98% of the ROC area of a human-based control, whereas the classical algorithms of Itti & Koch achieve only 84%.

read less

Abstract: A new bottom-up visual saliency model, Graph-Based Visual Saliency (GBVS), is proposed It consists of two steps: first forming activation maps on certain feature channels, and then normalizing them in a way which highlights conspicuity and admits combination with other maps The model is simple, and biologically plausible insofar as it is naturally parallelized This model powerfully predicts human fixations on 749 variations of 108 natural images, achieving 98% of the ROC area of a human-based control, whereas the classical algorithms of Itti & Koch ([2], [3], [4]) achieve only 84%

...read moreread less

Content maybe subject to copyright Report

Citations

PDF

Open Access

More filters

Journal Article•DOI•

ImageNet Large Scale Visual Recognition Challenge

[...]

Olga Russakovsky¹, Jia Deng², Hao Su¹, Jonathan Krause¹, Sanjeev Satheesh¹, Sean Ma¹, Zhiheng Huang¹, Andrej Karpathy¹, Aditya Khosla³, Michael S. Bernstein¹, Alexander C. Berg⁴, Li Fei-Fei¹ - Show less +8 more•Institutions (4)

Stanford University¹, University of Michigan², Massachusetts Institute of Technology³, University of North Carolina at Chapel Hill⁴

01 Dec 2015-International Journal of Computer Vision

TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.

...read moreread less

Abstract: The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the challenges of collecting large-scale ground truth annotation, highlight key breakthroughs in categorical object recognition, provide a detailed analysis of the current state of the field of large-scale image classification and object detection, and compare the state-of-the-art computer vision accuracy with human accuracy. We conclude with lessons learned in the 5 years of the challenge, and propose future directions and improvements.

...read moreread less

30,811 citations

Proceedings Article•DOI•

Frequency-tuned salient region detection

[...]

Radhakrishna Achanta¹, Sheila S. Hemami², Francisco J. Estrada¹, Sabine Süsstrunk¹•Institutions (2)

École Normale Supérieure¹, Cornell University²

20 Jun 2009

TL;DR: This paper introduces a method for salient region detection that outputs full resolution saliency maps with well-defined boundaries of salient objects that outperforms the five algorithms both on the ground-truth evaluation and on the segmentation task by achieving both higher precision and better recall.

...read moreread less

Abstract: Detection of visually salient image regions is useful for applications like object segmentation, adaptive compression, and object recognition. In this paper, we introduce a method for salient region detection that outputs full resolution saliency maps with well-defined boundaries of salient objects. These boundaries are preserved by retaining substantially more frequency content from the original image than other existing techniques. Our method exploits features of color and luminance, is simple to implement, and is computationally efficient. We compare our algorithm to five state-of-the-art salient region detection methods with a frequency domain analysis, ground truth, and a salient object segmentation application. Our method outperforms the five algorithms both on the ground-truth evaluation and on the segmentation task by achieving both higher precision and better recall.

...read moreread less

3,723 citations

Cites background or methods from "Graph-Based Visual Saliency"

...The saliency maps generated by most methods have low resolution [16, 22, 10, 7, 12]....
[...]
...Our method IG is compared against the five methods of IT [16], MZ [22], GB [10], SR [12], and AC [1] on 1000 images....
[...]
...Depending on the salient region detector, some maps additionally have ill-defined object boundaries [16, 10, 7], limiting their usefulness in certain applications....
[...]
...[10] create feature maps using Itti’s method but perform their normalization using a graph based approach....
[...]
...The DoG has also been used for interest point detection [21] and saliency detection [16, 10]....
[...]

Proceedings Article•DOI•

Global contrast based salient region detection

[...]

Ming-Ming Cheng¹, Guo-Xin Zhang¹, Niloy J. Mitra², Xiaolei Huang³, Shi-Min Hu¹ - Show less +1 more•Institutions (3)

Tsinghua University¹, King Abdullah University of Science and Technology², Lehigh University³

20 Jun 2011

TL;DR: This work proposes a regional contrast based saliency extraction algorithm, which simultaneously evaluates global contrast differences and spatial coherence, and consistently outperformed existing saliency detection methods.

...read moreread less

Abstract: Automatic estimation of salient object regions across images, without any prior assumption or knowledge of the contents of the corresponding scenes, enhances many computer vision and computer graphics applications. We introduce a regional contrast based salient object detection algorithm, which simultaneously evaluates global contrast differences and spatial weighted coherence scores. The proposed algorithm is simple, efficient, naturally multi-scale, and produces full-resolution, high-quality saliency maps. These saliency maps are further used to initialize a novel iterative version of GrabCut, namely SaliencyCut, for high quality unsupervised salient object segmentation. We extensively evaluated our algorithm using traditional salient object detection datasets, as well as a more challenging Internet image dataset. Our experimental results demonstrate that our algorithm consistently outperforms 15 existing salient object detection and segmentation methods, yielding higher precision and better recall rates. We also show that our algorithm can be used to efficiently extract salient object masks from Internet images, enabling effective sketch-based image retrieval (SBIR) via simple shape comparisons. Despite such noisy internet images, where the saliency regions are ambiguous, our saliency guided image retrieval achieves a superior retrieval rate compared with state-of-the-art SBIR methods, and additionally provides important target object region information.

...read moreread less

3,653 citations

Cites methods from "Graph-Based Visual Saliency"

...We have extensively evaluated our methods on publicly available benchmark data sets, and compared our methods with (eight) state-of-the-art saliency methods [17, 21, 32, 14, 15, 1, 2, 12] as well as with manually produced ground truth annotations1....
[...]
...(Left, middle) Different options of our method compared with GB[14], MZ[21], FT[2], IT[17], SR[15], AC[1], CA[12], and LC[32]....
[...]
...Following [2], we selected these methods according to: number of citations (IT[17] and SR[15]), recency (GB[14], SR, AC[1], FT[2] and CA[12]), variety (IT is biologically-motivated, MZ[21] is purely computational, GB is hybrid, SR works in the frequency domain, AC and FT output full resolution saliency maps), and being related to our approach (LC[32])....
[...]
...(a) original (b) IT[17] (c) MZ[21] (d) GB[14] (e) SR[15] (f) AC[1] (g) CA[12] (h) FT[2] (i) LC[32] (j) HC (k) RC Figure 2....
[...]
...[14] normalize the feature maps of Itti et al....
[...]

Journal Article•DOI•

Learning to Detect a Salient Object

[...]

Tie Liu¹, Zejian Yuan¹, Jian Sun², Jingdong Wang², Nanning Zheng¹, Xiaoou Tang³, Heung-Yeung Shum² - Show less +3 more•Institutions (3)

Xi'an Jiaotong University¹, Microsoft², The Chinese University of Hong Kong³

01 Feb 2011-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: A set of novel features, including multiscale contrast, center-surround histogram, and color spatial distribution, are proposed to describe a salient object locally, regionally, and globally.

...read moreread less

Abstract: In this paper, we study the salient object detection problem for images. We formulate this problem as a binary labeling task where we separate the salient object from the background. We propose a set of novel features, including multiscale contrast, center-surround histogram, and color spatial distribution, to describe a salient object locally, regionally, and globally. A conditional random field is learned to effectively combine these features for salient object detection. Further, we extend the proposed approach to detect a salient object from sequential images by introducing the dynamic salient features. We collected a large image database containing tens of thousands of carefully labeled images by multiple users and a video segment database, and conducted a set of experiments over them to demonstrate the effectiveness of the proposed approach.

...read moreread less

2,319 citations

Cites background from "Graph-Based Visual Saliency"

...Visual attention has been studied by researchers in physiology, psychology, neural systems, and computer vision for a long time....
[...]

Proceedings Article•DOI•

Saliency Detection via Graph-Based Manifold Ranking

[...]

Chuan Yang¹, Lihe Zhang¹, Huchuan Lu¹, Xiang Ruan², Ming-Hsuan Yang³ - Show less +1 more•Institutions (3)

Dalian University of Technology¹, Omron², University of California, Merced³

23 Jun 2013

TL;DR: This work considers both foreground and background cues in a different way and ranks the similarity of the image elements with foreground cues or background cues via graph-based manifold ranking, defined based on their relevances to the given seeds or queries.

...read moreread less

Abstract: Most existing bottom-up methods measure the foreground saliency of a pixel or region based on its contrast within a local context or the entire image, whereas a few methods focus on segmenting out background regions and thereby salient objects Instead of considering the contrast between the salient objects and their surrounding regions, we consider both foreground and background cues in a different way We rank the similarity of the image elements (pixels or regions) with foreground cues or background cues via graph-based manifold ranking The saliency of the image elements is defined based on their relevances to the given seeds or queries We represent the image as a close-loop graph with super pixels as nodes These nodes are ranked based on the similarity to background and foreground queries, based on affinity matrices Saliency detection is carried out in a two-stage scheme to extract background regions and foreground salient objects efficiently Experimental results on two large benchmark databases demonstrate the proposed method performs well when against the state-of-the-art methods in terms of accuracy and speed We also create a more difficult benchmark database containing 5,172 images to test the proposed saliency model and make this database publicly available with this paper for further studies in the saliency field

...read moreread less

2,278 citations

Cites background or methods from "Graph-Based Visual Saliency"

...We compare our method with fourteen state-of-the-art saliency detection algorithms: the IT [17], GB [14], MZ [25], SR [15], AC [1], Gof [11], FT [2], LC [37], RC [9], SVO [7], SF [27], CB [18], GS SP [34] and XIE [35] methods....
[...]
...[32] analyze multiple cues in a unified energy minimization framework and use a graph-based saliency model [14] to detect salient objects....
[...]
...We note that saliency models have been developed for eye fixation prediction [6, 14, 15, 17, 19, 25, 33] and salient object detection [1, 2, 7, 9, 23, 24, 32]....
[...]

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

Collapse

References

PDF

Open Access

More filters

Journal Article•DOI•

A model of saliency-based visual attention for rapid scene analysis

[...]

Laurent Itti¹, Christof Koch¹, Ernst Niebur²•Institutions (2)

California Institute of Technology¹, Johns Hopkins University²

01 Nov 1998-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: In this article, a visual attention system inspired by the behavior and the neuronal architecture of the early primate visual system is presented, where multiscale image features are combined into a single topographical saliency map.

...read moreread less

Abstract: A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented. Multiscale image features are combined into a single topographical saliency map. A dynamical neural network then selects attended locations in order of decreasing saliency. The system breaks down the complex problem of scene understanding by rapidly selecting, in a computationally efficient manner, conspicuous locations to be analyzed in detail.

...read moreread less

10,525 citations

Journal Article•DOI•

A saliency-based search mechanism for overt and covert shifts of visual attention.

[...]

Laurent Itti¹, Christof Koch¹•Institutions (1)

California Institute of Technology¹

01 Jun 2000-Vision Research

TL;DR: A detailed computer implementation of a saliency map scheme is described, focusing on the problem of combining information across modalities, here orientation, intensity and color information, in a purely stimulus-driven manner, which is applied to common psychophysical stimuli as well as to a very demanding visual search task.

...read moreread less

3,105 citations

"Graph-Based Visual Saliency" refers methods or result in this paper

...The parameters of this were checked against the literature [2] and [ 3 ], and were found to be almost identical, with a few slight alterations that actually improved performance relative to the published parameters....
[...]
...graph (ii) graph (iv) 0.981148 graph (i) graph (iv) 0.975313 graph (ii) I 0.974592 graph (ii) ave-max 0.974578 graph (ii) graph (iii) 0.974227 graph (i) graph (iii) 0.968414 self-info I 0.841054 *Bruce & Tsotsos [5] c-s DoG 0.840968 *Itti & Koch [ 3 ] c-s ave-max 0.840725 *Itti, Koch, &...
[...]
...This model powerfully predicts human �xations on 749 variations of 108 natural images, achieving 98% of the ROC area of a human-based control, whereas the classical algorithms of Itti & Koch ([2], [ 3 ], [4]) achieve only 84%....
[...]

Journal Article•DOI•

Modeling the role of salience in the allocation of overt visual attention.

[...]

Derrick Parkhurst¹, Klinton Law¹, Ernst Niebur¹•Institutions (1)

Johns Hopkins University¹

01 Jan 2002-Vision Research

TL;DR: In this paper, a biologically motivated computational model of bottom-up visual selective attention was used to examine the degree to which stimulus salience guides the allocation of attention in human eye movements while participants viewed a series of digitized images of complex natural and artificial scenes.

...read moreread less

1,417 citations

"Graph-Based Visual Saliency" refers background in this paper

...The standard approaches (e.g., [2], [9]) are based on biologically motivated feature selection, followed by center-surround operations which highlight local gradients, and nally a combination step leading to a "master map"....
[...]

Proceedings Article•

Saliency Based on Information Maximization

[...]

Neil D. B. Bruce¹, John K. Tsotsos¹•Institutions (1)

York University¹

05 Dec 2005

TL;DR: A model of bottom-up overt attention is proposed based on the principle of maximizing information sampled from a scene and is achieved in a neural circuit, which is demonstrated as having close ties with the circuitry existent in die primate visual cortex.

...read moreread less

Abstract: A model of bottom-up overt attention is proposed based on the principle of maximizing information sampled from a scene. The proposed operation is based on Shannon's self-information measure and is achieved in a neural circuit, which is demonstrated as having close ties with the circuitry existent in die primate visual cortex. It is further shown that the proposed salicney measure may be extended to address issues that currently elude explanation in the domain of saliency based models. Results on natural images are compared with experimental eye tracking data revealing the efficacy of the model in predicting the deployment of overt attention as compared with existing efforts.

...read moreread less

1,201 citations

"Graph-Based Visual Saliency" refers background or methods in this paper

...Recently, Bruce [5] and others [4] have hypothesized that fundamental quantities such as "self-information" and "surprise" are at the heart of saliency/attention....
[...]
...…activation: form an "activation map" (or maps) using the feature vectors (s3) normalization/combination: normalize the activation map (or maps, followed by a combination of the maps into a single map) In this light, [5] is a contribution to step (s2), whereas [4] is a contribution to step (s3)....
[...]
..."Improbable" would lead one to the formulation of Bruce [5], where a histogram of M(i; j) values is computed in some region around (i; j), subsequently normalized and treated as a probability distribution, so that A(i; j) = log(p(i; j)) is clearly de ned with p(i; j) = PrfM(i; j)jneighborhoodg: Another approach compares local "center" distributions to broader "surround" distributions and calls the Kullback-Leibler tension between the two "surprise" [4]....
[...]
...However, ultimately, Bruce computes a function which is additive in feature maps, with the main contribution materializing as a method of operating on a feature map in such a way to get an activation, or saliency, map....
[...]

Journal Article•DOI•

Preattentive texture discrimination with early vision mechanisms

[...]

Jitendra Malik¹, Pietro Perona¹•Institutions (1)

University of California, Berkeley¹

01 May 1990-Journal of The Optical Society of America A-optics Image Science and Vision

TL;DR: A model of human preattentive texture perception that can predict the salience of texture boundaries in any arbitrary gray-scale image and Quantitative predictions of the degree of discriminability of different texture pairs match well with experimental measurements of discriminateability in human observers.

...read moreread less

Abstract: We present a model of human preattentive texture perception. This model consists of three stages: (1) convolution of the image with a bank of even-symmetric linear filters followed by half-wave rectification to give a set of responses modeling outputs of V1 simple cells, (2) inhibition, localized in space, within and among the neural-response profiles that results in the suppression of weak responses when there are strong responses at the same or nearby locations, and (3) texture-boundary detection by using wide odd-symmetric mechanisms. Our model can predict the salience of texture boundaries in any arbitrary gray-scale image. A computer implementation of this model has been tested on many of the classic stimuli from psychophysical literature. Quantitative predictions of the degree of discriminability of different texture pairs match well with experimental measurements of discriminability in human observers.

...read moreread less

1,037 citations