scispace - formally typeset

Journal ArticleDOI

A compactness based saliency approach for leakages detection in fluorescein angiogram

01 Dec 2017-International Journal of Machine Learning and Cybernetics (Springer Berlin Heidelberg)-Vol. 8, Iss: 6, pp 1971-1979

TL;DR: This study has developed a novel saliency detection method based on compactness feature for detecting three common types of leakage in retinal fluorescein angiogram: large focal, punctate focal, and vessel segment leakage.

AbstractThis study has developed a novel saliency detection method based on compactness feature for detecting three common types of leakage in retinal fluorescein angiogram: large focal, punctate focal, and vessel segment leakage. Leakage from retinal vessels occurs in a wide range of retinal diseases, such as diabetic maculopathy and paediatric malarial retinopathy. The proposed framework consists of three major steps: saliency detection, saliency refinement and leakage detection. First, the Retinex theory is adapted to address the illumination inhomogeneity problem. Then two saliency cues, intensity and compactness, are proposed for the estimation of the saliency map of each individual superpixel at each level. The saliency maps at different levels over the same cues are fused using an averaging operator. Finally, the leaking sites can be detected by masking the vessel and optic disc regions. The effectiveness of this framework has been evaluated by applying it to different types of leakage images with cerebral malaria. The sensitivity in detecting large focal, punctate focal and vessel segment leakage is 98.1, 88.2 and 82.7 %, respectively, when compared to a reference standard of manual annotations by expert human observers. The developed framework will become a new powerful tool for studying retinal conditions involving retinal leakage.

Summary (2 min read)

1 Introduction

  • Fluorescein angiography (FA) is derived by taking a series of digital photographs of the retina before and after the fluorescein reaches the retinal circulation.
  • Three types of leakage can be observed on fluorescein angiogram (FA) of malaria retinopathy (MR): large focal, punctate focal, and vessel segment leakage [4], see Fig.
  • The saliency computation obtains a sparse representation of images, and an image can be classified as normal or abnormal (having bright lesions) by the obtained saliency information.
  • The limitation of these works are that the effectiveness and robustness is not convincing enough, such as the dataset used for validation in [12] is relative too small.
  • The leaking regions may be defined as salient regions as the leakage of fluorescent dye causes a large difference in brightness between the leak and surrounding non-leaking areas.

2 Method

  • The proposed framework consists of three main phases, which are saliency detection, saliency refinement, and leakage detection.
  • This is an advantage of the relative distance metric in reflecting the relative density of points and relative scale of the imaged objects.
  • Therefore, this section introduces an additional feature - compactness.
  • The measure of compactness of an object might be used as a complementary feature for saliency measurement, with the aim of overcoming the conflict of the falsely detected salient region.
  • Any small and/or isolated objects are eliminated after masking the vessel regions, by the use of a disk-shaped opening operation with a radius of 2 pixels.

3 Experimental Results

  • The detection framework in three types of leakage was implemented in Matlab 2013a.
  • The dataset the authors used comprises retinal FA images taken from children with CM admitted to the Malaria Research Project Ward, Queen Elizabeth Central Hospital, Blantyre, Malawi.
  • Consent was given by the guardians of subjects before examination and imaging.
  • 20 images (one per patient) with large focal leakage, and 10 images from 6 patients with punctate leakage were used.
  • The sensitivity, false positive per image, false negative per image, and ratio of overlapping area (OR) were used as the evaluation metrics.

3.2.1 Punctate focal

  • The top row of Fig. 4 show that the punctate focal leak regions were detected by both the human expert and the proposed method, respectively.
  • Table 1 illustrates the evaluation results of their automated method on detecting punctate focal leakage, in terms of sensitivity, false positives (FP) per image of punctate focal leakage number, and the false negatives (FN) per image of punctate focal leakage number.
  • According to the human reference standard there were 240 sites of punctate focal leakage in the tested 10 images.
  • 29 site were missed, leading to a false negative ratio of 2.832 per image.
  • Meanwhile, false positive ratio is 1.1 per image, because 11 sites were falsely determined as punctate focal leakage by proposed method due to the image artefact and local imbalanced illumination.

3.2.2 Large focal

  • The middle row of Fig. 4 demonstrates the detection of large focal leakage by both human expert and the proposed method, respectively.
  • It can be seen that the detected leakage regions are slightly different, since it is difficult to exactly define the boundary of a leaking area by hand since the contrast gradually fades at the edge of the lesion.
  • Table 1 also indicates the evaluation results of the developed framework in the detection of large focal leakage, in terms of sensitivity, false positives of focal leakage regions per image, false negatives per image of focal leakage regions, and the overlapping area between the regions detected by the proposed method and human expert.
  • In these studied images, a total of 41 sites of large focal leak were identified by the human expert reference standard.
  • In their framework, it only failed to detect 1 out of these focal leak sites, which has been evidenced by the sensitivity ratio of 0.976, and has a false negative ratio of 0.05 per image.

3.2.3 Vessel segment

  • The bottom of Fig. 4 indicate the results of vessel segment leakage detection.
  • An AUC of 1.0 means that the classifier distinguishes class examples perfectly.
  • In order to assess inter-observer variation, a third grader labeled the vessels using the same method.
  • In brief, this table shows that the developed automatic vessel segments leakage detection method can perform better than or at least as well as a human expert.

4 Conclusions

  • This paper proposed an innovative framework for the detection of three types of leakage founded on saliency detection, with the aim of supporting the study of abnormalities revealed in retinal images.
  • The framework benefits from three major components: saliency estimation, saliency refinement, and leakage detection.
  • The authors have successfully integrated compactness based feature to refined the intensity based saliency map.
  • The effectiveness of this method has been tested on FA images from patients with malarial retinopathy.
  • The authors method demonstrated satisfactory overall performance on large focal, punctate focal and vessel segment leakage detection.

Did you find this useful? Give us your feedback

...read more

Content maybe subject to copyright    Report

Noname manuscript No.
(will be inserted by the editor)
A Compactness based Saliency Approach for
Leakages Detection in Fluorescein Angiogram
Yitian Zhao · Pan Su · Jian Yang ·
Yifan Zhao · Yalin Zheng · Yongtian
Wang
Received: date / Accepted: date
Abstract This study has developed a novel saliency detection method based
on compactness feature for detecting three common types of leakage in retinal
fluorescein angiogram: large focal, punctate focal, and vessel segment leakage.
Leakage from retinal vessels occurs in a wide range of retinal diseases, such
as diabetic maculopathy and paediatric malarial retinopathy. The proposed
framework consists of three major steps: saliency detection, saliency refine-
ment and leakage detection. First, the Retinex theory is adapted to address
the illumination inhomogeneity problem. Then two saliency cues, intensity and
compactness, are proposed for the estimation of the saliency map of each indi-
vidual superpixel at each level. The saliency maps at different levels over the
same cues are fused using an averaging operator. Finally, the leaking sites can
be detected by masking the vessel and optic disc regions. The effectiveness of
this framework has been evaluated by applying it to different types of leakage
images with cerebral malaria. The sensitivity in detecting large focal, punc-
tate focal and vessel segment leakage is 98.1%, 88.2% and 82.7%, respectively,
when compared to a reference standard of manual annotations by expert hu-
man observers. The developed framework will become a new powerful tool for
studying retinal conditions involving retinal leakage.
Yitian Zhao, Jian Yang, and Yongtian Wang
Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Op-
tics and Electronics, Beijing Institute of Technology, Beijing, China
E-mail: yitian.zhao@bit.edu.cn
Pan Su
School of Control and Computer Engineering, North China Electric Power University, Baid-
ing, 071003, China
Yifan Zhao
BEPSRC Centre for Innovative Manufacturing in Through-life Engineering Services, Cran-
field University, Cranfield, UK
Yalin Zheng
Department of Eye and Vision Science, University of Liverpool, Liverpool, UK
International Journal of Machine Learning and Cybernetics, Vol. 8, Issue 6, December 2017, pp. 1971-1979
DOI:10.1007/s13042-016-0573-4
Published by Springer. This is the Author Accepted Manuscript issued with:
Creative Commons Attribution Non-Commercial License (CC:BY:NC 4.0).
The final published version (version of record) is available online at DOI:10.1007/s13042-016-0573-4.
Please refer to any applicable publisher terms of use.

2 Yitian Zhao et al.
Keywords Saliency · Compactness · Leakage · Fluorescein angiography ·
Malarial retinopathy
1 Introduction
Fluorescein angiography (FA) is derived by taking a series of digital pho-
tographs of the retina before and after the fluorescein reaches the retinal cir-
culation. FA’s ability to capture a range of retinal abnormalities in differential
diagnosis of retinal diseases, such as age-related macular degeneration (AMD),
diabetic retinopathy (DR) and malarial retinopathy (MR) [1–3]. FA provides
a map of retinal vascular structure and function by highlighting blockage to,
and leakage from, retinal vessels.
Retinal vessel leakage is particularly relevant to cerebral malaria (CM) [2,
3]. Three types of leakage can be observed on fluorescein angiogram (FA)
of malaria retinopathy (MR): large focal, punctate focal, and vessel segment
leakage [4], see Fig. 1. Large focal leakage describes one or more large, usually
circular, areas of leak, where the greatest linear diameter is larger than 125µm.
Punctate focal leak involves small but intensely bright sites of leak, where the
greatest linear diameter is less than 125µm. Leakage from vessel segments
appears as increased brightness and blurring of vessels. These three types of
leakage on MR recently have been defined on [5].
Fig. 1 Illustration of three types of leakage: (a) Large focal leakage. (b) Punctate focal
leakage. (c) Vessel segment leakage.
However, the research of detecting leakage on retinal image is relatively un-
explored. To the best of our knowledge, there is no automated method to detect
leakage on MR. In other ocular diseases has received little, such as diabetic
retinopathy [6], retinal vein occlusion [7], hyperfluorescent[8] and choroidal
neovascularization[9–11], if any, attention. In the section, only the related
works on detecting the abnormalities on different modalities of medical images
by saliency information will be reviewed. Yuan et al. [12] proposed a saliency
based ulcer detection method for wireless capsule endoscopy (WCE) diagno-
sis. It uses the multi-level superpixel representation as the pre-processing of
saliency detection, and the saliency map is generated by a fusion strategy of

Title Suppressed Due to Excessive Length 3
integrating all obtained saliency maps from all levels. This method is capable
to represent the accurate contour of the ulcer regions, and the ulcer regions
are located by classification tasks. Mahapatra and Sun used the saliency and
gradient information in Markov random field for non-rigid registration of dy-
namic MR cardiac perfusion images [13]. This approach addresses the problem
that majority of the nonrigid registration algorithms do not give satisfactory
results in the presence of intensity changes. A visual saliency based bright le-
sion detection is introduced in [14]. The spectral residual saliency model [15]
was employed to compute the saliency map of the color fundus retinal images.
The saliency computation obtains a sparse representation of images, and an
image can be classified as normal or abnormal (having bright lesions) by the
obtained saliency information. Jampani et al. [16] analyzed the relevance of
saliency models in detecting abnormalities in two types of medical images. The
authors extended the graph based visual saliency [17] models to detect the dif-
fuse lesions in chest X-ray images, and high contrast lesions in retinal images.
Zhao et al. [18] proposed a framework to detect the vessel abnormalities on
FA with application to malarial retinopathy. The authors used the intensity
and shape information to generate the saliency map to detect the intra vascu-
lar filling defect. However, the domain of uncertainty-based image processing
techniques may also be employed to yield the abnormality detection [19–24].
The limitation of these works are that the effectiveness and robustness is
not convincing enough, such as the dataset used for validation in [12] is relative
too small. The saliency provided a high quality of contrast enhanced images
by [13], but the gradient information still can be influenced by noise and does
not accurately register the boundary of the left ventricle. The approach [14] is
invalid on detection spot lesions, such as microneurysms.
1.1 Proposed Work
In this application, we define saliency in terms of information content: a key-
point corresponds to a particular image location within a structure with a low
probability of occurrence (i.e. high information content). For example, leakage
of fluorescent dye causes a large difference in brightness between the leak
and surrounding non-leaking areas. In this instance, the leaking regions may
be defined as salient regions as the leakage of fluorescent dye causes a large
difference in brightness between the leak and surrounding non-leaking areas.
We have successfully integrated the intensity and compactness information
to generate the saliency map for the detection of leakage in FA image. By
gathering the intensity and compactness features, the estimated saliency map
is not only able to distinguish the regions where the intensities are significant
different to their surroundings, but also has the capacity to avoid the non-
leaking regions with large intensity by considering the compactness of the
objects in the given images. The contributions of the proposed method can be
concluded as follows:

4 Yitian Zhao et al.
First, we propose a new unsupervised technique to detect and quantify the
type of leakage in MR by a novel adaptation of the concept of saliency [25].
Saliency is a predictor of object regions which attract human attention. Saliency
emerges from such characteristics in features of the image as visual unique-
ness, unpredictability, or rarity, and is often attributed to variations in specific
image attributes such as color, gradient, edges, and boundaries [26,27]. Such
attributes are characteristic of retinal leakage in FA images. It indicates the
relative importance of visual features, and is closely related to the charac-
teristics of human perception and processing of visual stimuli [25,28]. The
most general model of saliency detection is described by Itti and Koch [25].
Other existing saliency detection methods for feature determination can be
divided into four classes: pixel-based methods [29–31,25,32,17,33]; region-
based methods [26,34,28]; frequency-based methods [15,35–37]; parameter
learning-based methods [38–40].
Second, we have proposed a new feature - compactness to refine the intensity-
based saliency map. Normally, human observers pay more attention to a more
compact object than to a more diffuse object. The measure of compactness of
an object might therefore be of use as a complementary feature to intensity for
saliency measurement, with the aim of reducing the number of falsely-detected
salient regions.
The proposed leakage detection framework consists three main phases:
saliency estimation, saliency refinement, and leakage detection. The perfor-
mance of this framework also will be evaluated against the human expert
reference standard. The rest of this paper is organized as follows: section 2
describes the leakage detection methods, which consists of saliency detection
and refinement. A brief introduction to the dataset and evaluation metrics are
provided in section 3. In addition, the experimental results and the evalua-
tions against the human graders on detecting leakage are also demonstrated
in section 3. Finally, the paper is discussed and concluded in section 4.
2 Method
The proposed framework consists of three main phases, which are saliency
detection, saliency refinement, and leakage detection. In this section, all pro-
cedures are described in detail.
2.1 Saliency Detection
We now describe our saliency computation, which is based on the assumptions
that (i) a salient region is always different from its surrounding context [40],
and (ii) a more compact object tends to draw more visual attention [41].
The proposed saliency detection is formulated from Shannon’s information
definition. In the case of images, the content of a region around one pixel
would be more informative than a single pixel. Let P
i
I be the viable local

Title Suppressed Due to Excessive Length 5
Algorithm 1 Pseudo Code of Saliency and Leakage Detection
Input: An FA image I with focal leakage.
Saliency Detection:
1: for each image do
2: compute the intensity-based saliency using Eq. 3
3: end for
4: for each image do
5: compactness-based saliency using Eq. 6;
6: end for
Leakage Detection:
1: normalize S to [0, 1], threshold (T = 0.65) it to obtain ROIs.
2: for ROIs do
3: graph cut segmentation;
4: end for
5: mask the vessel region from S, and remove optic disc regions and small/isolated objects.
Output: The detected focal leaking areas in the given image I.
representation as a patch that represents pixel i (here, a 3× 3 window centred
on pixel i is used to define the patch), and I indicates the input image. The
patches can be seen as samples of a multivariate probability function (PDF).
The kernel density estimator (KDE) is chosen, as, being non-parametric, it
will permit the estimation of any PDF. The probability of a patch P
j
may
now be defined as
p(P
j
) =
1
Nh
X
iI
K(
d(P
j
, P
i
)
h
), (1)
where d is a distance function that will be discussed later, K is a kernel, h
is a smoothing parameter, and N represents the number of pixels. The KDE
method has the capacity to average out the contribution of each sample i
by spreading it over a certain area [42], which is defined by K. The multi-
variate distribution will have a higher probability if the patches are in dense
areas. From our experience, the most commonly used and appropriate kernel
is a Gaussian function with zero mean and standard deviation σ
k
. Using a
Gaussian kernel, equation (1) can be rewritten as
p
0
(P
j
) =
1
NΓ
X
iI
e
d
2
(P
j
,P
i
)
2σ
2
k
. (2)
The estimated probabilities are taken from an actual PDF by setting a
proper constant Γ . σ = 0.2 is chosen to substitute for h. After determining
the probability of the patches, the intensity-based saliency S measure can be
defined as follows:
S(j) = log
1
NΓ
X
iI
e
d
2
(P
j
,P
i
)
2σ
2
k
, (3)
where d is relative average distance. The relative distance is used in case the
distribution of the data is not uniform, and the distance metric mainly focuses

Citations
More filters

29 Jan 2020
TL;DR: A conditional generative adversarial network (GAN) - based method to directly learn the mapping relationship between structure fundus images and fundus fluorescence angiography images is proposed and local saliency maps, which define each pixel's importance, are used to define a novel saliency loss in the GAN cost function.
Abstract: Fluorescein angiography can provide a map of retinal vascular structure and function, which is commonly used in ophthalmology diagnosis, however, this imaging modality may pose risks of harm to the patients. To help physicians reduce the potential risks of diagnosis, an image translation method is adopted. In this work, we proposed a conditional generative adversarial network(GAN) - based method to directly learn the mapping relationship between structure fundus images and fundus fluorescence angiography images. Moreover, local saliency maps, which define each pixel's importance, are used to define a novel saliency loss in the GAN cost function. This facilitates more accurate learning of small-vessel and fluorescein leakage features.

2 citations


Cites methods from "A compactness based saliency approa..."

  • ...Most of the commonly used methods (Zhao et al., 2017, 2015) to generate saliency map of FFA image are pixel-based methods, which are time-consuming and thus do not meet the requirements of our work....

    [...]


Posted Content
Abstract: Fluorescein angiography can provide a map of retinal vascular structure and function, which is commonly used in ophthalmology diagnosis, however, this imaging modality may pose risks of harm to the patients To help physicians reduce the potential risks of diagnosis, an image translation method is adopted In this work, we proposed a conditional generative adversarial network(GAN) - based method to directly learn the mapping relationship between structure fundus images and fundus fluorescence angiography images Moreover, local saliency maps, which define each pixel's importance, are used to define a novel saliency loss in the GAN cost function This facilitates more accurate learning of small-vessel and fluorescein leakage features

1 citations


References
More filters

Journal ArticleDOI
Abstract: A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented. Multiscale image features are combined into a single topographical saliency map. A dynamical neural network then selects attended locations in order of decreasing saliency. The system breaks down the complex problem of scene understanding by rapidly selecting, in a computationally efficient manner, conspicuous locations to be analyzed in detail.

9,639 citations


01 Jan 1998
TL;DR: A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented, which breaks down the complex problem of scene understanding by rapidly selecting conspicuous locations to be analyzed in detail.

8,566 citations


"A compactness based saliency approa..." refers background in this paper

  • ...The most general model of saliency detection is described by Itti and Koch [25]....

    [...]

  • ...KochC,UllmanC (1985)Shifts in selective visual attention: towards the underlying neural circuitry....

    [...]

  • ...Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis....

    [...]

  • ...adaptation of the concept of saliency [25]....

    [...]

  • ...It indicates the relative importance of visual features, and is closely related to the characteristics of human perception and processing of visual stimuli [25, 28]....

    [...]


Book ChapterDOI
TL;DR: This study addresses the question of how simple networks of neuron-like elements can account for a variety of phenomena associated with this shift of selective visual attention and suggests a possible role for the extensive back-projection from the visual cortex to the LGN.
Abstract: Psychophysical and physiological evidence indicates that the visual system of primates and humans has evolved a specialized processing focus moving across the visual scene. This study addresses the question of how simple networks of neuron-like elements can account for a variety of phenomena associated with this shift of selective visual attention. Specifically, we propose the following: (1) A number of elementary features, such as color, orientation, direction of movement, disparity etc. are represented in parallel in different topographical maps, called the early representation. (2) There exists a selective mapping from the early topographic representation into a more central non-topographic representation, such that at any instant the central representation contains the properties of only a single location in the visual scene, the selected location. We suggest that this mapping is the principal expression of early selective visual attention. One function of selective attention is to fuse information from different maps into one coherent whole. (3) Certain selection rules determine which locations will be mapped into the central representation. The major rule, using the conspicuity of locations in the early representation, is implemented using a so-called Winner-Take-All network. Inhibiting the selected location in this network causes an automatic shift towards the next most conspicious location. Additional rules are proximity and similarity preferences. We discuss how these rules can be implemented in neuron-like networks and suggest a possible role for the extensive back-projection from the visual cortex to the LGN.

3,795 citations


Proceedings ArticleDOI
20 Jun 2011
TL;DR: This work proposes a regional contrast based saliency extraction algorithm, which simultaneously evaluates global contrast differences and spatial coherence, and consistently outperformed existing saliency detection methods.
Abstract: Automatic estimation of salient object regions across images, without any prior assumption or knowledge of the contents of the corresponding scenes, enhances many computer vision and computer graphics applications. We introduce a regional contrast based salient object detection algorithm, which simultaneously evaluates global contrast differences and spatial weighted coherence scores. The proposed algorithm is simple, efficient, naturally multi-scale, and produces full-resolution, high-quality saliency maps. These saliency maps are further used to initialize a novel iterative version of GrabCut, namely SaliencyCut, for high quality unsupervised salient object segmentation. We extensively evaluated our algorithm using traditional salient object detection datasets, as well as a more challenging Internet image dataset. Our experimental results demonstrate that our algorithm consistently outperforms 15 existing salient object detection and segmentation methods, yielding higher precision and better recall rates. We also show that our algorithm can be used to efficiently extract salient object masks from Internet images, enabling effective sketch-based image retrieval (SBIR) via simple shape comparisons. Despite such noisy internet images, where the saliency regions are ambiguous, our saliency guided image retrieval achieves a superior retrieval rate compared with state-of-the-art SBIR methods, and additionally provides important target object region information.

3,388 citations


"A compactness based saliency approa..." refers background or methods in this paper

  • ...However, it is observed that using the intensity feature alone to detect the salient region is not always successful [26], also has been evidenced by Fig....

    [...]

  • ...Other existing saliency detection methods for feature determination can be divided into four classes: pixel-based methods [17, 25, 29–33]; region-based methods [26, 28, 34]; frequency-based methods [15, 35–37]; parameter learning-based methods [38–40]....

    [...]


Proceedings Article
04 Dec 2006
TL;DR: A new bottom-up visual saliency model, Graph-Based Visual Saliency (GBVS), is proposed, which powerfully predicts human fixations on 749 variations of 108 natural images, achieving 98% of the ROC area of a human-based control, whereas the classical algorithms of Itti & Koch achieve only 84%.
Abstract: A new bottom-up visual saliency model, Graph-Based Visual Saliency (GBVS), is proposed It consists of two steps: first forming activation maps on certain feature channels, and then normalizing them in a way which highlights conspicuity and admits combination with other maps The model is simple, and biologically plausible insofar as it is naturally parallelized This model powerfully predicts human fixations on 749 variations of 108 natural images, achieving 98% of the ROC area of a human-based control, whereas the classical algorithms of Itti & Koch ([2], [3], [4]) achieve only 84%

3,335 citations


Frequently Asked Questions (2)
Q1. What are the contributions in "A compactness based saliency approach for leakages detection in fluorescein angiogram" ?

This study has developed a novel saliency detection method based on compactness feature for detecting three common types of leakage in retinal fluorescein angiogram: large focal, punctate focal, and vessel segment leakage. The effectiveness of this framework has been evaluated by applying it to different types of leakage images with cerebral malaria. The developed framework will become a new powerful tool for studying retinal conditions involving retinal leakage. 

In their future work, the authors will extend the current saliency-guided leakage detection model to other vascular diseases, such as diabetic retinopathy. The authors believe that this innovative framework has the potential to be developed further as a useful tool for fast, accurate and objective assessment of leak in a range of vascular diseases.