![Fig. 4. Computing the local spectrum [31]. A specific ROI on an IVUS image is shown in (a). The raw RF signals along 5 adjacent A-lines comprising this ROI are graphically represented in (b). The individual spectrum corresponding to these RF signals are shown in (c). The average of these 5 spectra yields the spectrum shown in (d). This local spectrum encodes information about the tissue within the ROI.](/figures/fig-4-computing-the-local-spectrum-31-a-specific-roi-on-an-z5p8mmj4.webp)
![Fig. 1. (Left) RF ablation catheter in the heart; (Right) An example of lesion created by RF ablation [15], where the red line indicates the change in US image upon energy delivery.](/figures/fig-1-left-rf-ablation-catheter-in-the-heart-right-an-lwgh6l8u.webp)
![Fig. 2. (Left) Standard cross-sectional IVUS image in cartesian coordinates, and (Right) its corresponding polar representation: ρ represents the depth in the tissue and θ the position (angle) in the rotation of the probe [35].](/figures/fig-2-left-standard-cross-sectional-ivus-image-in-cartesian-3gemee4x.webp)
![Fig. 5. Samples of RF signals collected over time from a fixed spot of tissue under emission of sequential ultrasound form one RF time series [88].](/figures/fig-5-samples-of-rf-signals-collected-over-time-from-a-fixed-20x7z3o3.webp)
![Fig. 3. An example of a lesion in orthogonal planes of an automated 3D breast ultrasound volume [54].](/figures/fig-3-an-example-of-a-lesion-in-orthogonal-planes-of-an-2s507kvf.webp)
Aberystwyth University
Ultrasound tissue classification
Shan, Caifeng; Tan, Tao; Han, Jungong; Huang, Di
Published in:
Artificial Intelligence Review
Publication date:
2020
Citation for published version (APA):
Shan, C., Tan, T., Han, J., & Huang, D. (2020). Ultrasound tissue classification: A review. Artificial Intelligence
Review.
General rights
Copyright and moral rights for the publications made accessible in the Aberystwyth Research Portal (the Institutional Repository) are
retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the
legal requirements associated with these rights.
• Users may download and print one copy of any publication from the Aberystwyth Research Portal for the purpose of private study or
research.
• You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the Aberystwyth Research Portal
Take down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.
tel: +44 1970 62 2400
email: is@aber.ac.uk
Download date: 09. Aug. 2022
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 13, NO. 9, SEPTEMBER 2014 1
Ultrasound Tissue Classification: A Review
Caifeng Shan, Tao Tan, Jungong Han, Di Huang
Abstract— Ultrasound (US) imaging is the most widespread
medical imaging modality for creating images of the human body
in clinical practice. Tissue classification in ultrasound has been
established as one of the most active research areas, driven by
many important clinical applications. In this paper, we present
a survey on ultrasound tissue classification, focusing on recent
advances in this area. We start with a brief review on the main
clinical applications. We then introduce the traditional approaches,
where the existing research on feature extraction and classifier de-
sign are reviewed. As deep learning approaches becoming popular
for medical image analysis, the recent deep learning methods for
tissue classification are also introduced. We briefly discuss the
FDA-cleared techniques being used clinically. We conclude with
the discussion on the challenges and research focus in future.
Index Terms— Tissue Classification, Tissue Characteri-
zation, Machine Learning, Deep Learning, Ultrasound Im-
age Analysis
I. INTRODUCTION
Different medical imaging modalities are available and widely used
nowadays in clinical practice to create images of the human body,
such as computed tomography (CT), magnetic resonance imaging
(MR), positron emission tomography (PET), and ultrasound (US).
Among those, US imaging is the most widespread modality for
visualizing human soft tissue, because of its advantages compared to
others: cheap, harmless (no ionizing radiations), allowing real-time
feedback, convenient to operate, well established technology present
in all places, and so on. On the other hand, because of the limited
field of view, shadows, speckle noise, and other artifacts in the US
images, the interpretation of US images is sometimes difficult.
One main target of US image (signal) analysis is tissue classifica-
tion. Ultrasound tissue classification is to analyze the characteristics
of the US data and their correlation to the pathological state of
tissue, and design a classifier to distinguish the US data into different
tissue types (or states). Tissue classification in ultrasound has many
important applications, such as cancer diagnosis (in prostate, breast,
liver, etc.) and cardiovascular disease diagnosis and intervention.
Driven by the unmet clinical needs to distinguish different tissue
types (e.g., healthy versus diseased) in US, tissue characterization
and classification have received much attention in recent years [1],
[2].
Traditionally the process of tissue classification can be divided
into two main steps: 1) feature extraction: in this step, ultrasound
modelling and signal analysis techniques are used to extract char-
acteristic features from US image (signal) that can describe and
hopefully differentiate different tissue types [1]. These characteristics
may not be visible to human eyes, and it is necessary to examine
the ultrasound image. If the features are discriminative, different
C. Shan is with College of Electrical Engineering and Automation,
Shandong University of Science and Technology, Qingdao 266590,
China
T. Tan is with Eindhoven University of Technology, Eindhoven 5600
MB, The Netherlands
J. Han is with Department of Computer Science, Aberystwyth Univer-
sity, SY23 3DB, UK
D. Huang is with Beijing Advanced Innovation Center for Big Data and
Brain Computing, Beihang University, Beijing 100191, China
C. Shan (caifeng.shan@gmail.com) is the corresponding author.
tissue types will be represented as separate clusters in the feature
space. Unfortunately in practice most extracted features have low
discriminative power. Therefore, in order to discriminate different
tissue types, a proper classifier is needed. This is step 2): classifier
design, which aims to define the optimal decision boundary in the
feature space to separate different tissue types. The classifier can
be trained by applying machine learning and pattern recognition
techniques on the data with ground truth. Different techniques have
been exploited for ultrasound tissue classification.
Since 2012, deep learning algorithms such as convolutional neural
networks (CNNs) [3] have become a powerful tool for automatically
classifying pixels (patches) in the images. It usually contains several
pairs of a convolution layer and a pooling layer. These layers behave
as feature extractors. The intermediate outputs of these layers are
fully connected to a multi-layer perception neural network for the
classification task. New techniques including dropout [4], batch
normalization [5] and resnetblock [6] were proposed to improve
performance of neural networks. Deep learning approaches have been
extensively exploited for medical image analysis, including tissue
classification in ultrasound.
Ultrasound tissue classification is difficult, because the interaction
between biological tissue (an inhomogeneous medium) and acoustic
wave is very hard to model [1]. Although the quality of ultrasound
images, in terms of signal-to-noise and contrast-to-noise ratios, has
been improved substantially in recent years, it remains a challenging
task to classify different tissue types in US images. This paper
attempts to provide a review on ultrasound tissue classification,
particularly focusing on recent advances in this area. The work on
tissue characterization prior to 2009 was reviewed in the articles [1],
[2].
The paper is organized as follows. We start with a brief review
on the main clinical applications in Section II. We then introduce
the existing research on feature extraction and classifier design in
Section III and Section IV respectively. The recent deep learning
approaches for tissue classification are discussed in Section V.
Section VI introduces the FDA-cleared machine learning algorithms
being used clinically. Section VII discusses the challenges and
research directions in future, and finally Section VIII concludes the
paper.
II. CLINICAL APPLICATIONS
Medical ultrasound has a very broad range of clinical uses. Ultra-
sound tissue classification can be applied in many clinical fields, for
instance, tissue classification plays an important role in ultrasound-
based cancer diagnosis, e.g., by classifying the tissue regions as
benign or malignant. Here we briefly introduce the main clinical
applications that have received the most attention in the recent
literature.
A. Cardiology
The primary aim of non-invasive cardiac imaging is to provide
information on the diagnosis and severity of underlying cardiac
conditions [7]. Echocardiography (ultrasound imaging of the heart)
is the most common cardiac imaging procedure performed in clinical
practice, due to its portability, low cost, and patient acceptance.
2 JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 13, NO. 9, SEPTEMBER 2014
Echocardiography has also been one of the driving application areas
of medical ultrasound [8]. Cardiac imaging techniques characterise
the underlying tissue directly, by assessing a signal from the tissue
itself, or indirectly, by inferring tissue characteristics from global
or regional function [7]. Although Cardiac Magnetic Resonance
(CMR) imaging currently is the most investigated modality for tissue
characterisation, this technology is difficult to scale up for routine use.
On the other hand, echocardiography remains the primary imaging
tool for most patients, so it represents an attractive alternative for
cardiac tissue characterization.
Traditionally, integrated backscatter, which measures the ultrasonic
reflectivity of the region of interest, was a major focus of tissue
characterisation research [9]. However, backscatter has limited ability
to reflect fibrosis in those with lower levels of myocardial fibrosis,
such as coronary artery disease [10]. In [11], the frequency content
from echocardiography and spectral analysis techniques were inves-
tigated for differentiating three different cardiac tissue types (cardiac
adipose tissue, myocardium, and blood). Recently texture features
of myocardium have been extracted from still ultrasound images
for tissue characterization [12]. Speckle tracking echocardiography
(STE) is a technique used to assess myocardial deformation at both
segmental and global levels. Since distinct myocardial pathologies
affect deformation differently, information about the underlying tissue
can be inferred by STE. While other modalities such as CMR assess
tissue characteristics through changes in the acquired myocardial tis-
sue images, STE deformation parameters assess the impact of under-
lying pathology on tissue function. The available studies correlating
STE deformation parameters with underlying tissue characteristics are
reviewed in [7]. The most commonly used deformation measurements
are those of strain (the change in length compared to initial length),
strain rate (strain divided by time), and rotation (or twist). Machine
learning and deep learning methods have been explored with STE
features for tissue classification [13], [14].
In recent years ultrasound has been increasingly used for image-
guided cardiac interventions or therapy [16]–[18]. Cardiac ablation,
to create a set of transmural lesions in cardiac tissue, has been the
main therapy method for atrial fibrillation. In thermal ablation, the
target tissue is coagulated by transferring heat to the target area.
Radio-frequency (RF) ablation and the lesion created by RF ablation
are illustrated in Figure 1. One challenge for ablation therapy is the
monitoring of the temperature rise and the extend of the ablated
region in the tissue. US-based techniques for evaluating heat-induced
lesions have been studied recently [15], [19], [20]. In an earlier
work [21], both the textural and spectral features were considered
for analyzing the ablated regions in the tissue. Methods have also
been developed to estimate the temperature increase during the
ablation process by detecting the change of sound speed, attenuation
coefficient, and backscattering. In [19], US imaging is employed to
investigate the temporal evolution and spatial extent of the lesion
created by the HIFU (high intensity focused ultrasound) ablation. It
is shown that spectral analysis of RF signals, which is related to the
physical scatter properties, can potentially be used for monitoring the
evolution of HIFU lesions. Imani et al. [20], [22] proposed to classify
ablated tissue using RF time series features. The acoustical coefficient
of nonlinearity is estimated in [23] for discriminating tissues during
cardiac ablation therapy.
B. Vascular Disease
Cardiovascular diseases are responsible for a third of all deaths
in women worldwide and more than a half in men [24]. Ultrasound
imaging has become one of the most important modalities used in
the assessment of vascular diseases. Virtually all peripheral arterial
and venous structures can be visualized with the duplex ultrasound
(DU), which currently is the main diagnostic modality used in deep
venous thrombosis and carotid disease [25]. For carotid artery disease,
the carotid plaque is traditionally judged according to the degree
of stenosis and a 70% or greater diameter loss is an indication for
surgery. Later it was recognized that not only must the degree of
stenosis be evaluated, but also the carotid plaque instability, as it
is an important determinant of stroke risk [26]. The DU allows for
the study of plaque constituents; for instance, fibrotic tissue, which
renders the plaque more stable, has different brightness from lipids.
Traditionally the carotid plaque composition is determined based on
the pixel brightness, i.e., using a threshold-based method. In [26],
a multi-scale descriptor was used for pixel-level tissue classification
in DU images. Similarly, various image features were considered in
[27], [28] for plaque classification. A recent review on carotid artery
ultrasound image analysis can be found in [29].
In the era of atherosclerosis, intravascular ultrasound (IVUS), a
catheter-based imaging technique, has evolved as a valuable technique
for diagnosis and intervention for coronary disease, by providing
more precise measurement on intimal thickness and vulnerable
plaques. In contrast, angiography, the traditionally used gold-standard
in the imaging of vascular morphology, can only depict contrast
agent filled lumen and not the vessel wall [25]. IVUS provides
images of vascular structure by scanning inside the vessel, which
are acquired by a mechanically rotated transducer or a multi-element
transducer array. For the latter case, an array of transducers is
disposed around the probe, with the synchronized emission of US
waves. The imaging plane, perpendicular to the long axis of the
catheter, provides a 360 degree image of the vessel. An example
IVUS image is shown in Figure 2. By converting the A-lines from
polar to cartesian coordinates, the full circumference of the vessel
wall can be visualized. Therefore, all components of the vessel are
visualized: the cross sectional luminal size, shape and vessel wall, as
well as the various layers of the wall. IVUS has been used to guide
the intervention by analyzing the vessel condition, e.g., assessing
the atherosclerotic plaque amount and composition. IVUS can also
be used to check the vessel condition after the intervention, and to
monitor the status of disease over the time.
In IVUS images, calcifications are demonstrated as hyperechoic
areas, whereas hemorrage or fat deposition inside an atheromatic
plaque is hypoechoic. Subsequently, the plaque can be classified
as lipid, calcified and fibrous, according to its acoustic properties
[30] Tissue classification in IVUS images can automatically predict
vulnerable plaques as well as quantify the amount of the different
tissues; many approaches have been proposed in literature [31]–[36],
which will be discussed in detail in the following sections.
C. Breast Cancer
Breast cancer is the most common cancer in women globally,
and early detection is the key to reduce the death rate. Women
with dense breasts have a risk of breast cancer four to six times
higher than that of women with no or little dense tissue [37]. To
improve cancer detection in dense breasts, personalized breast cancer
screening with ultrasound has been proposed for women with dense
breasts and women with elevated risk factors for developing breast
cancer. Supplemental breast ultrasound (BUS) cancer screening can
detect small, early stage invasive cancers that appear to be occult
on mammograpms due to breast density [38]. Furthermore, BUS is
more convenient and safer for clinical use [39] and it can be used
as an alternative screening device to mammography for women with
harmful mutations in either BRCA1 (breast cancer gene type1) or
BRCA2 since no radiation is involved. Berg et al. [40] found that the
SHELL shan et al.: ULTRASOUND TISSUE CLASSIFICATION: A REVIEW 3
Fig. 1. (Left) RF ablation catheter in the heart; (Right) An example of lesion created by RF ablation [15], where the red line indicates the change
in US image upon energy delivery.
Fig. 2. (Left) Standard cross-sectional IVUS image in cartesian coordinates, and (Right) its corresponding polar representation: ρ represents the
depth in the tissue and θ the position (angle) in the rotation of the probe [35].
supplemental yield of using ultrasound together with mammography
was 4.2 cancers per 1000 women screened. Kaplan et al. [41]
found 6 (0.36%) extra cancers from 1862 mammgraphic negative
patients with dense breasts. However handheld ultrasound breast
cancer screening is operator dependent, uneasy to reproduce, time
consuming and relatively expensive as a screening procedure when
performed by radiologists.
Tissue classification in BUS aims to distinguish benign masses
(cysts and fibroadenomas) and malignant cancerous masses. Various
approaches have been investigated to detect the suspicious masses in
B-mode BUS images [42]–[44], where the challenge is to characterize
the textured appearance and geometry of a tumor relative to normal
tissue. A survey on cancer detection and classification in BUS images
can be found in [39]. Recently, automated 3D breast ultrasound
system (ABUS) as a novel modality was proposed to overcome the
drawbacks of the traditional 2D ultrasound. Fig. 3 shows one example
of an ABUS image. It usually involves a heavy compression from
a membrane on the breast. With a swipe of a wide linear or curved
transducer, a number of transversal images are generated and recon-
structed to become a 3D volume. Different than 2D ultrasound, ABUS
provides the possibility of visualizing speculation patterns associated
with malignancy on coronal planes. Because of the standard imaging
procedure, it is possible to perform temporal analysis on prior and
current exams. Since the ABUS images are standardly defined, many
researcher paid lots of attention on tumor classification [45]–[49]
and cancer detection [50]–[57] using various techniques. Currently
GE invenia ABUS system is FDA-cleared for screening purpose [58]
while Siemens ABVS system is FDA-cleared for diagnosis purpose
[59].
Besides ABUS system which uses reflected echoes, transmission
ultrasound is employed for cancer detection and diagnosis as well by
FDA-cleared systems such as SoftVue System [60] from Delphinus
Medical Technologies and QTscan [61] from QT ultrasound. These
systems usually employ rotating ultrasound transducer, to capture
details of the tissue of the uncompressed breast for patients positioned
in the prone position. Images are generated using both reflection and
transmission modalities. The addition of tissue attributes of sound
speed and attenuation can enrich diagnostic information for tissue
classification.
D. Prostate Cancer
Prostate cancer is the second most common cancer worldwide and
the most common malignancy in men [62]. It is due to the abnormal
and uncontrolled cell mutation and replication in the prostate gland.
4 JOURNAL OF L
A
T
E
X CLASS FILES, VOL. 13, NO. 9, SEPTEMBER 2014
Fig. 3. An example of a lesion in orthogonal planes of an automated
3D breast ultrasound volume [54].
As with any cancer, early detection and treatment is vital for good
survival rates of prostate cancer. Although multi-parametric MRI
has been increasingly used for prostate cancer diagnosis, transrectal
ultrasound (TRUS) is widely used for evaluation of the prostate,
because of its advantages of low costs, good availability, and ability
to visualize the prostate in real time [63]. TRUS has been used for the
early detection and active surveillance of prostatic cancer. Currently,
serum prostate-specific antigen (PSA) and Digital Rectal Examination
(DRE) are used for screening. If either study is abnormal, TRUS-
guided biopsy is performed for diagnostic confirmation [64].
By analyzing the characteristics of the tissue regions, tissue clas-
sification in TRUS provides a malignancy map to guide biopsy,
which can reduce the number of unnecessary biopsy. However,
the prostate regions in TRUS images are characterized by a weak
texture, speckle, short gray scale ranges, and shadow regions [8];
detection and delineation of prostate pathology in TRUS is difficult
due to the heterogeneous and multi-focal nature of prostatic lesions.
Tissue classification has been extensively studied for prostate can-
cer detection [65]. The earlier work on TRUS tissue classification
discriminated the rectangular regions around biopsy needle insertion
points with textural features [66]. Later different types of features
were considered together with more sophisticated classifiers [67]–
[69], for instance, textural features and spectral parameters extracted
from RF data were employed in [67]. Morphologic features and mul-
tiresolution textural features were also used for malignancy detection
[69]. Non-linear Higher Order Spectra (HOS) features and Discrete
Wavelet Transform (DWT) coefficients were considered in [70]. It
has been shown that combining features extracted from RF analysis
of ultrasound signals and texture would result in better classification
[65]. HistoScanning [71]–[73] is a commercially-available ultrasound
tissue characterisation technique that has shown encouraging results
in the detection of prostate cancer.
In addition to gray-scale ultrasound, color Doppler ultrasound
(CDUS) has also been used for the evaluation of prostate cancer
by detecting the increased perfusion compared with surrounding
prostate tissue [63]. It has been shown that the combination of
CDUS and gray-scale ultrasound can detect a greater number
of prostate cancers than gray-scale ultrasound alone [74]. New
ultrasound techniques such as contrast-enhanced ultrasound (CEUS)
and ultrasound elastography [74], [75], have been developed to
improve the detection of prostate cancer. MRI-ultrasound fusion is
another novel imaging technique that is being used to guide prostate
biopsy, where the target lesion is marked on the prebiopsy MRI and
fused to the real-time TRUS images [76].
The previous two subsections have discussed two main application
areas in cancer diagnosis, breast and prostate. Ultrasound tissue
classification has also been investigated for diagnosis of other cancer
types, for instance, early detection of liver tumour [77]–[80]. Focal
liver lesions such as cysts and tumours are concentrated over a quite
small area of the tissue and are difficult to identify. The classification
of lesions in ultrasound liver image usually depends heavily on the
characteristics of the lesions including internal echo, morphology,
edge, echogenicity, and posterior echo enhancement. In [77], texture
features are used to distinguish malignant and benign liver tumours.
Automatic classification of thyroid tissue into benign and malignant
types using US are investigated in [81]–[83]; a review on thyroid
cancer tissue characterization can be seen in [84].
III. FEATURE EXTRACTION
The traditional approaches to ultrasound tissue classification start
with extracting discriminative features from the ultrasound signal or
image. In the US image formation process [1], the radio-frequency
(RF) signal acquired by the US transducer undergoes filtering,
envelop detection, log compression, and post-processing to finally
give a grayscale representation (which is often called A-line). The
grayscale signal is then interpolated and rasterized to give a B-mode
or M-model image for display.
A large amount of features of different nature have been investi-
gated in the literature for tissue characterization and classification (as
summarized in Table I). Here we group them into three categories.
The first category relies on tissue appearance in US images, where
texture and morphology are usually analyzed. In the second category,
the RF signal or envelope-detected data is analyzed. Spectral analysis
is often performed to extract characteristics about the behavior
(property) of different tissue types in the frequency domain. The last
category is to fuse and combine different types of features.
A. Image-based approaches
Before extracting features from US images, image pre-processing
and image segmentation are usually performed. Pre-processing
consists of speckle reduction and image enhancement (see [39] for
some details), and image segmentation is to segment the image into
Regions of Interest (ROI) for further analysis [8].
Texture features — Feature extraction from US images aims at
describing the appearance of tissue. Most studies have focused on
textural properties of speckle which represent the macroscopic ap-
pearance of scattering generated by tissue micro-structures [121]. By
identifying spatial variations in pixel intensities and quantifying them
into numerical features, texture analysis has been widely adopted and
different kinds of texture features are available in the literature: Gabor
filter responses, derivatives of Gaussian filters, wavelet transform, co-
occurrence matrices, Local Binary Patterns, fractal spaces, Markov
random fields, and so on. Here we list the texture features that have
been widely used for US tissue classification
• Grey Level Co-occurrence Matrix (GLCM) [122]: GLCM is
a well-known statistical tool for extracting texture information
from images. It measures how often different combinations of
pixel intensities occur in an image. GLCM can also be defined
as an estimation of the joint probability density function of gray