Noname manuscript No.
(will be inserted by the editor)
A Compactness based Saliency Approach for
Leakages Detection in Fluorescein Angiogram
Yitian Zhao · Pan Su · Jian Yang ·
Yifan Zhao · Yalin Zheng · Yongtian
Wang
Received: date / Accepted: date
Abstract This study has developed a novel saliency detection method based
on compactness feature for detecting three common types of leakage in retinal
fluorescein angiogram: large focal, punctate focal, and vessel segment leakage.
Leakage from retinal vessels occurs in a wide range of retinal diseases, such
as diabetic maculopathy and paediatric malarial retinopathy. The proposed
framework consists of three major steps: saliency detection, saliency refine-
ment and leakage detection. First, the Retinex theory is adapted to address
the illumination inhomogeneity problem. Then two saliency cues, intensity and
compactness, are proposed for the estimation of the saliency map of each indi-
vidual superpixel at each level. The saliency maps at different levels over the
same cues are fused using an averaging operator. Finally, the leaking sites can
be detected by masking the vessel and optic disc regions. The effectiveness of
this framework has been evaluated by applying it to different types of leakage
images with cerebral malaria. The sensitivity in detecting large focal, punc-
tate focal and vessel segment leakage is 98.1%, 88.2% and 82.7%, respectively,
when compared to a reference standard of manual annotations by expert hu-
man observers. The developed framework will become a new powerful tool for
studying retinal conditions involving retinal leakage.
Yitian Zhao, Jian Yang, and Yongtian Wang
Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Op-
tics and Electronics, Beijing Institute of Technology, Beijing, China
E-mail: yitian.zhao@bit.edu.cn
Pan Su
School of Control and Computer Engineering, North China Electric Power University, Baid-
ing, 071003, China
Yifan Zhao
BEPSRC Centre for Innovative Manufacturing in Through-life Engineering Services, Cran-
field University, Cranfield, UK
Yalin Zheng
Department of Eye and Vision Science, University of Liverpool, Liverpool, UK
International Journal of Machine Learning and Cybernetics, Vol. 8, Issue 6, December 2017, pp. 1971-1979
DOI:10.1007/s13042-016-0573-4
Published by Springer. This is the Author Accepted Manuscript issued with:
Creative Commons Attribution Non-Commercial License (CC:BY:NC 4.0).
The final published version (version of record) is available online at DOI:10.1007/s13042-016-0573-4.
Please refer to any applicable publisher terms of use.
2 Yitian Zhao et al.
Keywords Saliency · Compactness · Leakage · Fluorescein angiography ·
Malarial retinopathy
1 Introduction
Fluorescein angiography (FA) is derived by taking a series of digital pho-
tographs of the retina before and after the fluorescein reaches the retinal cir-
culation. FA’s ability to capture a range of retinal abnormalities in differential
diagnosis of retinal diseases, such as age-related macular degeneration (AMD),
diabetic retinopathy (DR) and malarial retinopathy (MR) [1–3]. FA provides
a map of retinal vascular structure and function by highlighting blockage to,
and leakage from, retinal vessels.
Retinal vessel leakage is particularly relevant to cerebral malaria (CM) [2,
3]. Three types of leakage can be observed on fluorescein angiogram (FA)
of malaria retinopathy (MR): large focal, punctate focal, and vessel segment
leakage [4], see Fig. 1. Large focal leakage describes one or more large, usually
circular, areas of leak, where the greatest linear diameter is larger than 125µm.
Punctate focal leak involves small but intensely bright sites of leak, where the
greatest linear diameter is less than 125µm. Leakage from vessel segments
appears as increased brightness and blurring of vessels. These three types of
leakage on MR recently have been defined on [5].
Fig. 1 Illustration of three types of leakage: (a) Large focal leakage. (b) Punctate focal
leakage. (c) Vessel segment leakage.
However, the research of detecting leakage on retinal image is relatively un-
explored. To the best of our knowledge, there is no automated method to detect
leakage on MR. In other ocular diseases has received little, such as diabetic
retinopathy [6], retinal vein occlusion [7], hyperfluorescent[8] and choroidal
neovascularization[9–11], if any, attention. In the section, only the related
works on detecting the abnormalities on different modalities of medical images
by saliency information will be reviewed. Yuan et al. [12] proposed a saliency
based ulcer detection method for wireless capsule endoscopy (WCE) diagno-
sis. It uses the multi-level superpixel representation as the pre-processing of
saliency detection, and the saliency map is generated by a fusion strategy of
Title Suppressed Due to Excessive Length 3
integrating all obtained saliency maps from all levels. This method is capable
to represent the accurate contour of the ulcer regions, and the ulcer regions
are located by classification tasks. Mahapatra and Sun used the saliency and
gradient information in Markov random field for non-rigid registration of dy-
namic MR cardiac perfusion images [13]. This approach addresses the problem
that majority of the nonrigid registration algorithms do not give satisfactory
results in the presence of intensity changes. A visual saliency based bright le-
sion detection is introduced in [14]. The spectral residual saliency model [15]
was employed to compute the saliency map of the color fundus retinal images.
The saliency computation obtains a sparse representation of images, and an
image can be classified as normal or abnormal (having bright lesions) by the
obtained saliency information. Jampani et al. [16] analyzed the relevance of
saliency models in detecting abnormalities in two types of medical images. The
authors extended the graph based visual saliency [17] models to detect the dif-
fuse lesions in chest X-ray images, and high contrast lesions in retinal images.
Zhao et al. [18] proposed a framework to detect the vessel abnormalities on
FA with application to malarial retinopathy. The authors used the intensity
and shape information to generate the saliency map to detect the intra vascu-
lar filling defect. However, the domain of uncertainty-based image processing
techniques may also be employed to yield the abnormality detection [19–24].
The limitation of these works are that the effectiveness and robustness is
not convincing enough, such as the dataset used for validation in [12] is relative
too small. The saliency provided a high quality of contrast enhanced images
by [13], but the gradient information still can be influenced by noise and does
not accurately register the boundary of the left ventricle. The approach [14] is
invalid on detection spot lesions, such as microneurysms.
1.1 Proposed Work
In this application, we define saliency in terms of information content: a key-
point corresponds to a particular image location within a structure with a low
probability of occurrence (i.e. high information content). For example, leakage
of fluorescent dye causes a large difference in brightness between the leak
and surrounding non-leaking areas. In this instance, the leaking regions may
be defined as salient regions as the leakage of fluorescent dye causes a large
difference in brightness between the leak and surrounding non-leaking areas.
We have successfully integrated the intensity and compactness information
to generate the saliency map for the detection of leakage in FA image. By
gathering the intensity and compactness features, the estimated saliency map
is not only able to distinguish the regions where the intensities are significant
different to their surroundings, but also has the capacity to avoid the non-
leaking regions with large intensity by considering the compactness of the
objects in the given images. The contributions of the proposed method can be
concluded as follows:
4 Yitian Zhao et al.
First, we propose a new unsupervised technique to detect and quantify the
type of leakage in MR by a novel adaptation of the concept of saliency [25].
Saliency is a predictor of object regions which attract human attention. Saliency
emerges from such characteristics in features of the image as visual unique-
ness, unpredictability, or rarity, and is often attributed to variations in specific
image attributes such as color, gradient, edges, and boundaries [26,27]. Such
attributes are characteristic of retinal leakage in FA images. It indicates the
relative importance of visual features, and is closely related to the charac-
teristics of human perception and processing of visual stimuli [25,28]. The
most general model of saliency detection is described by Itti and Koch [25].
Other existing saliency detection methods for feature determination can be
divided into four classes: pixel-based methods [29–31,25,32,17,33]; region-
based methods [26,34,28]; frequency-based methods [15,35–37]; parameter
learning-based methods [38–40].
Second, we have proposed a new feature - compactness to refine the intensity-
based saliency map. Normally, human observers pay more attention to a more
compact object than to a more diffuse object. The measure of compactness of
an object might therefore be of use as a complementary feature to intensity for
saliency measurement, with the aim of reducing the number of falsely-detected
salient regions.
The proposed leakage detection framework consists three main phases:
saliency estimation, saliency refinement, and leakage detection. The perfor-
mance of this framework also will be evaluated against the human expert
reference standard. The rest of this paper is organized as follows: section 2
describes the leakage detection methods, which consists of saliency detection
and refinement. A brief introduction to the dataset and evaluation metrics are
provided in section 3. In addition, the experimental results and the evalua-
tions against the human graders on detecting leakage are also demonstrated
in section 3. Finally, the paper is discussed and concluded in section 4.
2 Method
The proposed framework consists of three main phases, which are saliency
detection, saliency refinement, and leakage detection. In this section, all pro-
cedures are described in detail.
2.1 Saliency Detection
We now describe our saliency computation, which is based on the assumptions
that (i) a salient region is always different from its surrounding context [40],
and (ii) a more compact object tends to draw more visual attention [41].
The proposed saliency detection is formulated from Shannon’s information
definition. In the case of images, the content of a region around one pixel
would be more informative than a single pixel. Let P
i
∈ I be the viable local
Title Suppressed Due to Excessive Length 5
Algorithm 1 Pseudo Code of Saliency and Leakage Detection
Input: An FA image I with focal leakage.
Saliency Detection:
1: for each image do
2: compute the intensity-based saliency using Eq. 3
3: end for
4: for each image do
5: compactness-based saliency using Eq. 6;
6: end for
Leakage Detection:
1: normalize S to [0, 1], threshold (T = 0.65) it to obtain ROIs.
2: for ROIs do
3: graph cut segmentation;
4: end for
5: mask the vessel region from S, and remove optic disc regions and small/isolated objects.
Output: The detected focal leaking areas in the given image I.
representation as a patch that represents pixel i (here, a 3× 3 window centred
on pixel i is used to define the patch), and I indicates the input image. The
patches can be seen as samples of a multivariate probability function (PDF).
The kernel density estimator (KDE) is chosen, as, being non-parametric, it
will permit the estimation of any PDF. The probability of a patch P
j
may
now be defined as
p(P
j
) =
1
Nh
X
i∈I
K(
d(P
j
, P
i
)
h
), (1)
where d is a distance function that will be discussed later, K is a kernel, h
is a smoothing parameter, and N represents the number of pixels. The KDE
method has the capacity to average out the contribution of each sample i
by spreading it over a certain area [42], which is defined by K. The multi-
variate distribution will have a higher probability if the patches are in dense
areas. From our experience, the most commonly used and appropriate kernel
is a Gaussian function with zero mean and standard deviation σ
k
. Using a
Gaussian kernel, equation (1) can be rewritten as
p
0
(P
j
) =
1
NΓ
X
i∈I
e
d
2
(P
j
,P
i
)
2σ
2
k
. (2)
The estimated probabilities are taken from an actual PDF by setting a
proper constant Γ . σ = 0.2 is chosen to substitute for h. After determining
the probability of the patches, the intensity-based saliency S measure can be
defined as follows:
S(j) = −log
1
NΓ
X
i∈I
e
−
d
2
(P
j
,P
i
)
2σ
2
k
, (3)
where d is relative average distance. The relative distance is used in case the
distribution of the data is not uniform, and the distance metric mainly focuses