FAST FULLY AUTOMATIC BRAIN DETECTION IN FETAL MRI
USING DENSE ROTATION INVARIANT IMAGE DESCRIPTORS
Bernhard Kainz
1∗
Kevin Keraudren
1
Vanessa Kyriakopoulou
2
Mary Rutherford
2
Joseph V. Hajnal
2
Daniel Rueckert
1
1
Biomedical Image Analysis Group, Department of Computing, Imperial College London, UK
2
Department Biomedical Engineering Division of Imaging Sciences, King’s College London, UK
ABSTRACT
Automatic detection of the fetal brain in Magnetic Resonance
(MR) Images is especially difficult due to arbitrary orienta-
tion of the fetus and possible movements during the scan. In
this paper, we propose a method to facilitate fully automatic
brain voxel classification by means of rotation invariant vol-
ume descriptors. We calculate features for a set of 50 prenatal
fast spin echo T2 volumes of the uterus and learn the appear-
ance of the fetal brain in the feature space. We evaluate our
novel classification method and show that we can localize the
fetal brain with an accuracy of 100% and classify fetal brain
voxels with an accuracy above 97%. Furthermore, we show
how the classification process can be used for a direct seg-
mentation of the brain by simple refinement methods within
the raw MR scan data leading to a final segmentation with a
Dice score above 0.90.
Index Terms— fetal MRI reconstruction, fetal brain localiza-
tion, fetal brain segmentation
1. INTRODUCTION
Magnetic resonance imaging (MRI) of the fetus has been
shown to be a useful tool for accurate prenatal diagnostics
and to assess fetal development. Currently, mainly the brain
and the whole fetus appearance are qualitatively examined
in clinical practice [1, 2]. Fetal motion and its unpredictable
nature (Fig. 2) put high demand on radiologists and make an
automatic evaluation of the scan challenging.
In this paper we propose rotation invariant volume descrip-
tors [3] for medical imaging in combination with state-of-the-
art machine learning methods to provide a robust detection
of the fetal brain. This detection process produces a reliable
classification of brain voxels. It can simplify the application
of advanced motion correction methods [4], which produce a
high resolution volume from consecutive single-shot fast spin
echo (SSFSE) T2-weighted scans. Alternatively, it can serve
as a starting point for automatic or interactive segmentation.
To demonstrate the feasibility of our method, we chose the lat-
ter application additionally to the localization challenge, and
show that we can use the probability map resulting from the
localization process for a basic brain segmentation.
∗
Bernhard Kainz is supported by a Marie Curie Intra-European Fellow-
ship (FP7-PEOPLE-2012-IEF F.A.U.S.T. 325661).
Related work: Fetal MRI is a relatively new field, with
little work published on fully automatic processing of these
datasets. In [5, 6], 3D template matching is used to detect the
eyes, enabling a subsequent 2D/3D graph-cut segmentation
to extract the brain. This approach based on 3D rigid tem-
plates lacks the flexibility necessary to deal with motion arti-
facts as well as fetal malformations. The methods proposed
in [7] and [8] address the variability of fetal MRI through ma-
chine learning. In [7], a Random Forest classifier first distin-
guishes between maternal and fetal tissues before identifying
different tissues of the fetal head, while [8] combines prior
knowledge of the fetal head size with MSER detection and a
bag-of-words model. In contrast to [7], which obtains rotation
invariance by rotating the training data and [8], which focuses
on 2D slice detection, our method operates fully in 3D space,
learning rotation invariant features, and is likely to be faster
than all the methods proposed so far.
Contributions: To determine the orientation of the fetus, an
automatic detection could try to register the scan to a known
reference coordinate frame. This, however, is infeasible due
to the influence of maternal tissues and the non-rigid nature of
the fetus as a whole. A different approach is to derive image
descriptors that are similar in any orientation of the image.
Descriptors with such properties are called rotation invariant.
We propose a novel way to apply rotation invariant feature
descriptors based on Spherical Harmonics to medical volume
images and show their potential to solve the orientation prob-
lem. The fetus is not sedated and may move during the vol-
ume acquisition, leading to motion artifacts between planes.
By choosing the right trade-off between feature description
accuracy and robustness of the descriptor, we can minimize
the influence of motion artifacts on the detection process.
Fetuses develop rapidly and organ shapes and internal appear-
ance change drastically over time. Our test dataset contains
50 fetuses aged between 22–37 weeks. Our brain classifica-
tion method must be insensitive to local changes in appear-
ance, which can be achieved by a correct choice of the feature
descriptor parameters to average out small local details while
preserving the general appearance in feature space. By sim-
ple post-processing of the detection probability map, we also
show how to obtain a basic segmentation of the fetal brain.
2. METHOD
Fig. 1 provides an overview of our brain detection pipeline.
Labeled
Scan 1...n
Normalize and
NLM denoising
Feature-space
transformation
Combine feature
vector
Train Regression
Forest classifier
Test
scan
Normalize and
NLM denoising
Predict with
classifier 3D
Result
2D region
refinement
Feature-space
transformation
Fig. 1. Method overview: after normalization and non-local
median (NLM) denoising, each training image is transformed
to feature space and feature vectors are learned by a Random
Forest. The resulting classifier is used to detect the brain from
a new dataset, its probability map providing a rough initial
segmentation which is later used in a 2D refinement.
Image acquisition: Images are acquired on a Philips Achieva
1.5T scanner, the mother lying either on the back or on the
side in order to feel comfortable. A typical acquisition begins
with a localizer scan, which is used to align the scan main axis
approximately parallel to the fetus. Single-shot fast spin echo
(SSFSE) T2-weighted sequences are used to acquire a stack
of images of the mother’s womb. Each acquisition of a 2D
image takes approximately 0.5–1.0s, which makes through-
plane movement artifacts very likely until the whole image
stack is available. Several of these image stacks are usually
acquired, parallel and perpendicular to the fetus’ main axis.
This allows to account for movement artifacts during later
post-processing steps [4]. In this work, we use one randomly
selected image stack from each subject to show our method’s
ability to cope with through-plane movement artifacts. Ex-
ample slices from our datasets are shown in Fig. 2. All fetal
subjects have gestational ages between 22–37 weeks.
(a) (b)
Fig. 2. Example slices from our training data: (a) exemplary
slices: The fetus can be randomly oriented. The through-
plane resolution is low and moving artifacts can occur dur-
ing the acquisition of the slices as shown in the multi-planar
reconstruction (MPR) in (b).
Transformation to a rotation invariant feature space:
Before transforming the image to feature space, we we nor-
malize the image intensities and apply a non-local median
denoising filter [9]. We define rotation invariant features
with the angular power spectrum ||a
0l
||
2
of the expansion
coefficients a
l
of Spherical Harmonic Functions similar to
Skibbe et al. [3]. Skibbe et al. evaluate several different
basis functions and show strong evidence that the power
spectrum of the expansion with spherical 3D Gabor basis
functions, represented by a superposition of Bessel functions
B
l
s
(k), are highly suitable for both classification accuracy
and computation time when used for medical 3D images.
Thus, we also define our spherical 3D Gabor descriptors
(SGD) coefficients at an expansion l, scale t and frequency
k with a
k
l
(x) = (
√
t
−k
)
l
∇
l
(I ? B
0
s
(k))(x). The transforma-
tion into the Gabor domain can be realized with one single
initial cross correlation of the image and the basis function
I ? B
0
s
(k) followed by an iterative application of the spherical
up-derivative operator ∇
l
, which defines the spherical coun-
terpart of a gradient operator in conventional calculus [3].
A major difference between the work by Skibbe et al. and
ours is that our feature vectors do not describe structures as a
whole but rather small patches of the organ. We also imple-
mented our own version based on Nvidia CUDA [10] and ac-
celerated the algorithm to achieve an approximately 60 times
faster feature space transformation.
Furthermore, we determined the best trade-off between SGD
detail and computation performance using cross-validation on
two randomly selected datasets that have been excluded from
the training and testing datasets. We found the optimal expan-
sion bandwidth to be around 20 derivatives.
Cross-validation was also used to find the best size of the
voxel set covered by one descriptor. In contrast to Skibbe et
al., we do not describe the whole object (organ) in harmonics
space, but a small subset of the organ at every position in the
volume. We obtained the best results with descriptor sizes be-
tween 3−5mm. We combine features at the minimum and the
maximum of this range to form a feature vector consisting of
coefficients for a smaller and a larger spherical neighborhood
to gain additional descriptive power. Our final SGD covers
three different frequencies to represent the local image struc-
ture at each scale. We use k = 0, π, and 2π. Therefore, each
SGD feature descriptor vector has a length of 66 elements.
Learning of brain features: To improve learning perfor-
mance and maintain a spherical shape of the descriptor in
the feature space, the data is subsampled in-plane to match
the through-plane resolution, so that the input image has an
isotropic voxel size.
We train a state-of-the-art Classification Forest ensemble
learning method based on decision trees for the SGD image
features [11, 12], because of its trade-off between efficiency
and classification performance. All descriptors labeled as
brain and a third of the background descriptors to reduce
training time are used for learning.
To tune the decision tree preference toward the brain features,
the weights of misclassified brain voxels are altered accord-
ing to the average ratio of the size of the fetal brain compared
to the 3D volume of the whole scan. In our datasets, approxi-
mately 10% of the voxels belong to the brain and 90% belong
to background. A maximum of 256 trees have been used, with
a maximum depth of 256. During cross-validation, these val-
ues resulted in the best trade-off between computation time
and classification performance. An example probability map
of the prediction for a test-image is presented in Fig. 3(a). Us-
ing the largest region with the highest probabilities (p > 0.8)
results in a highly accurate area within the fetal brain.
Fetal brain segmentation refinement: To demonstrate the
feasibility of our method to serve as a basis for further post-
processing, we use the probability map from the Classifica-
tion Forest as input for a slice based brain segmentation. We
use 2D slices parallel to the two smallest voxel sides for this
example because of the large through-plane resolution of our
datasets (4mm) and the small volume of the fetus.
The probability map is first thresholded at a value higher than
0.5. While outlying regions may have probabilities above
0.5, the values for true brain voxels are generally higher. To
remove possible remaining outlying regions, the largest 3D
connected component is selected. During our experiments,
this approach always resulted in an area within the fetal brain
(localization accuracy = 100%). For each 2D slice, the min-
imum (min) and maximum (max) intensity values of the in-
tersection of the thresholded probability map with the slice
is computed. Taking advantage of the bright appearance of
brain pixels in T2 MRI, all pixels on a slice whose intensity
value is smaller than min − 0.1 · (max − min) are set to the
minimum of the whole volume.
Finally, a 2D level-set [13] is initialized from the thresholded
probability map and evolved on the processed slices. A few
hundred iterations are usually sufficient for an accurate seg-
mentation of the fetal brain.
(a) (b) (c) (d)
Fig. 3. The probability map for a test dataset is colored red
the higher the probability for a brain voxel is (a). (b) shows
the detection result, which is the input for the segmentation
refinement process after thresholding the probability map and
selection of the largest connected region. (c) shows the final
result after simple refinement, and (d) shows the ground truth
expert segmentation. We use [14] for visualization.
3. RESULTS
We have tested our method on 50 datasets semi-automatically
segmented using manually initialized graph cuts. The datasets
have been cropped to an average size of 200 × 200 × 70 vox-
els encompassing the whole fetal body in order to reduce the
number of background voxels and therefore the computation
time during testing. Note that our method shows the same per-
formance without cropping but at higher computational costs.
We have randomly picked ten times ten different datasets as
test set and learned the appearance of the brain in the feature
space from the remaining 40 datasets. Tab. 1 summarizes the
averaged results for the detection process, considering values
above 0.5 in the probability map as correct classification. The
probability in the center regions of the brain is usually > 0.85,
which can be taken as threshold value if localization is the
target application. During our experiments, the centroid of
the largest area classified with a brain probability above 0.85
showed a 100% localization accuracy.
Using a probability threshold above 0.5 reaches a classifica-
tion accuracy of over 97% on average without refinement.
However, since most of the voxels in a dataset represent back-
ground information, a method predicting always zero would
also have a high accuracy. Therefore, we also evaluated the
weighted average of Precision and Recall with the DICE coef-
ficient (DC) in Tab. 1. An average DC of 0.850% before seg-
mentation refinement shows that our base method is already
highly robust and that it shows only a few to no outliers.
We evaluate the refined brain segmentation according to [15]
in Tab. 1. Considering the low through-plane resolution of
our datasets, the average distance values show that we can
reliably segment the brain in most slices. The maximum dis-
tance errors show that our refinement method has limitations
in regions with large anatomical abnormalities, where an un-
usually large gradient will stop the level set evolution, and for
slices where only a few voxels were detected as brain (border
slices). The level set over-segments in these peripheral slices
because of the absence of a clear gradient as well as partial
volume effects from the skull. It produces a region which
is on average 5 − 10mm too large. These border slices are
mainly responsible for higher average and maximum surface
distance errors and could be left out in practice.
Figure 3 shows an example result from our test-dataset. Our
method has been evaluated on an Intel Xeon E5-2643 system
with 16 GB RAM and an Nvidia Titan Graphics card. The
transformation to feature space (GPU) and voxel classifica-
tion (CPU) takes on average 1.4s. The segmentation refine-
ment (CPU) takes another 5.7s. Therefore, our approach can
be executed within approximately 7s.
Note that our method provides the same 100% localization ac-
curacy as [8]. However, our approach is faster to compute and
provides additionally an accurate brain segmentation mask.
4. CONCLUSION AND FUTURE WORK
We have presented a fast, reliable and fully automatic way to
localize the fetal brain from prenatal volumetric MRI scans.
Our approach allows the automatic classification of a set of
voxels within the fetal brain with high precision and recall.
While previous methods were limited to the localization of
the brain, we also show that our detection process can serve
as a suitable prior for further fully automatic segmentation of
the fetal brain. Our method is fast enough for a near real-time
application to the scan during the acquisition process and can
provide essential input for subsequent scanning procedures.
Voxel classification Segmentation refinement
Experiment Accuracy Precision Recall DC RAVD ASSD RMSSSD MSSD DC
1 0.98 0.87 0.84 0.85 2.91 4.21 11.42 21.42 0.89
2 0.97 0.88 0.82 0.85 2.64 4.15 10.53 24.53 0.91
3 0.96 0.83 0.81 0.82 2.73 4.52 9.31 19.52 0.92
4 0.97 0.89 0.82 0.85 3.31 5.03 11.42 28.42 0.89
5 0.98 0.90 0.83 0.86 2.84 4.30 10.49 23.04 0.89
6 0.95 0.84 0.82 0.83 2.99 4.24 12.82 22.75 0.90
7 0.98 0.87 0.86 0.87 2.61 4.15 11.99 20.42 0.91
8 0.97 0.88 0.86 0.87 2.31 4.17 10.42 17.53 0.92
9 0.98 0.87 0.86 0.87 2.44 4.21 9.62 20.21 0.93
10 0.98 0.86 0.83 0.84 2.95 4.33 11.83 22.01 0.91
Average 0.971 0.868 0.833 0.850 2.773 4.331 10.985 21.985 0.907
Table 1. Voxel classification, left: Performance results for the brain classification method. These measures reflect the clas-
sifier performance without any post-processing. Using the simplest refinement method from Section 2 – selecting the largest
connected region – would lead to a mean DC > 0.88. Segmentation refinement, right: Results from our segmentation refine-
ment process for our validation experiment. The table shows measures according to [15]: relative absolute volume difference
(RAVD), average symmetric surface distance (ASSD), root mean square symmetric surface distance (RMSSSD), maximum
symmetric surface distance (MSSD), and average Dice coefficient (DC).
http://goo.gl/uKNqXf shows a video about our method.
5. REFERENCES
[1] M. Rutherford, S. Jiang, J. Allsop, L. Perkins, L. Srini-
vasan, T. Hayat, S. Kumar, and J. Hajnal, “MR imaging
methods for assessing fetal brain development,” Dev
Neurobiol, vol. 68, no. 6, pp. 700–11, 2008.
[2] C. Limperopoulos and C. Clouchoux, “Advancing fetal
brain MRI: targets for the future,” Seminars in perina-
tology, vol. 33, no. 4, pp. 289–298, Aug. 2009.
[3] H. Skibbe, M. Reisert, T. Schmidt, T. Brox, O. Ron-
neberger, and H. Burkhardt, “Fast Rotation Invariant 3D
Feature Computation utilizing Efficient Local Neighbor-
hood Operators,” IEEE Transactions on Pattern Analy-
sis and Machine Intelligence, vol. 34, no. 8, pp. 1563 –
1575, Aug 2012.
[4] M. Kuklisova-Murgasova, G. Quaghebeur, M. A.
Rutherford, J. V. Hajnal, and J. A. Schnabel, “Re-
construction of fetal brain MRI with intensity matching
and complete outlier removal,” Elsevier Medical Image
Analysis, vol. 16, no. 8, pp. 1550 – 1564, 2012.
[5] J. Anquez, E. D. Angelini, and I. Bloch, “Automatic seg-
mentation of head structures on fetal MRI,” in in Proc.
IEEE International Symposium on Biomedical Imaging.
2009, pp. 109–112, IEEE Press.
[6] Y. Taleb, M. Schweitzer, C. Studholme, M. Koob, J.-
L. Dietemann, and F. Rousseau, “Automatic Template-
based Brain Extraction in Fetal MR Images,” Abstract
hal-00879089, 6 2013.
[7] M. Ison, R. Donner, E. Dittrich, G. Kasprian, D. Prayer,
and G. Langs, “Fully Automated Brain Extraction and
Orientation in Raw Fetal MRI,” in Proc. of MICCAI
Workshop on Paediatric and Perinatal Imaging, 2012.
[8] K. Keraudren, V. Kyriakopoulou, M. Rutherford, J.V.
Hajnal, and D. Rueckert, “Localisation of the Brain in
Fetal MRI Using Bundled SIFT Features,” in MICCAI
2013, vol. 8149 of LNCS, pp. 582–589. 2013.
[9] Y. Li and S. Osher, “A new median formula with appli-
cations to PDE based denoising,” Commun. Math. Sci.,
vol. 7, no. 3, pp. 741–753, 2009.
[10] NVIDIA Corporation, NVIDIA CUDA Compute Uni-
fied Device Architecture Programming Guide, NVIDIA
Corporation, 2012.
[11] A. Criminisi and J. Shotton, Decision Forests for Com-
puter Vision and Medical Image Analysis, Springer-
Verlag New York Incorporated, 2013.
[12] G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal
of Software Tools, 2000.
[13] R. Malladi, J.A. Sethian, and B.C. Vemuri, “Shape
Modeling with Front Propagation: a Level Set Ap-
proach,” IEEE Transactions on Pattern Analysis and
Machine Intelligence, vol. 17, no. 2, pp. 158–175, 1995.
[14] I. Wolf, M. Vetter, I. Wegner, T. B
¨
ottger, M. Nolden,
M. Sch
¨
obinger, M. Hastenteufel, T. Kunert, and H.-P.
Meinzer, “The Medical Imaging Interaction Toolkit.,”
Elsevier Medical Image Analysis, vol. 9, no. 6, pp. 594–
604, 2005.
[15] B. Van Ginneken, T. Heimann, and M. Styner, “3D
segmentation in the clinic: A grand challenge,” in In:
MICCAI Workshop on 3D Segmentation in the Clinic: A
Grand Challenge., 2007.
![Fig. 3. The probability map for a test dataset is colored red the higher the probability for a brain voxel is (a). (b) shows the detection result, which is the input for the segmentation refinement process after thresholding the probability map and selection of the largest connected region. (c) shows the final result after simple refinement, and (d) shows the ground truth expert segmentation. We use [14] for visualization.](/figures/fig-3-the-probability-map-for-a-test-dataset-is-colored-red-269v6i9r.webp)
![Table 1. Voxel classification, left: Performance results for the brain classification method. These measures reflect the classifier performance without any post-processing. Using the simplest refinement method from Section 2 – selecting the largest connected region – would lead to a mean DC > 0.88. Segmentation refinement, right: Results from our segmentation refinement process for our validation experiment. The table shows measures according to [15]: relative absolute volume difference (RAVD), average symmetric surface distance (ASSD), root mean square symmetric surface distance (RMSSSD), maximum symmetric surface distance (MSSD), and average Dice coefficient (DC).](/figures/table-1-voxel-classification-left-performance-results-for-27vekopi.webp)