User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability

TL;DR: The methods and software engineering philosophy behind this new tool, ITK-SNAP, are described and the results of validation experiments performed in the context of an ongoing child autism neuroimaging study are provided, finding that SNAP is a highly reliable and efficient alternative to manual tracing.

About: This article is published in NeuroImage.The article was published on 2006-07-01 and is currently open access. It has received 6669 citations till now. The article focuses on the topics: Scale-space segmentation & Image segmentation.

Summary

1 Introduction

• Segmentation is used to measure the size and shape of brain structures, to guide spatial normalization of anatomy between individuals and to plan medical intervention.
• Many brain research laboratories continue to use manual delineation as the technique of choice for image segmentation.
• The authors suspect that insufficient attention to developing tools that make parameter selection intuitive has prevented semiautomatic methods from replacing manual delineation as the tool of choice in the clinical research environment.
• SNAP is available free of charge both as a stand-alone application that can be installed and executed quickly and as source code that can be used to derive new software.
• A short overview of automatic image segmentation, as well as some popular medical imaging tools that support it, is given in Sec. 2. A brief summary of active contour segmentation and level set methods appears in Sec. 3.1. Sec. 3.2 highlights the main features of SNAP’s user interface and software architecture.

2 Previous Work

• In many clinical laboratories, biomedical image segmentation involves having a trained expert delineate the boundaries of anatomical structures in consecutive slices of 3D images.
• A validation of caudate nucleus segmentation by Gurleyik and Haacke (2002) reports the interrater reliability of 0.84.
• On the other side of the segmentation spectrum lie fully automated methods based on probabilistic models of image intensity, atlas deformation and statistical shape models.
• This type of registration assumes one-to-one correspondence between subject anatomies, which is not always the case, considering high variability in cortical folding and potential presence of pathology.
• These tools carry a steep learning curve, due the large number of features that they provide.

3.1 Active Contour Evolution

• SNAP implements two well-known 3D active contour segmentation methods: Geodesic Active Contours by Caselles et al. (1993, 1997) and Region Competition by Zhu and Yuille (1996).
• I denotes convolution of I with the isotropic Gaussian kernel with aperture σ; and ν and λ are user-supplied parameters that determine the shape of the monotonic mapping between the normalized gradient magnitude and the speed function, illustrated in Fig.
• Zhu and Yuille (1996) compute the external force by estimating the probability that a voxel belongs to the structure of interest and the probability that it belongs to the background at each voxel in the input image.
• In SNAP, these probabilities are estimated using fuzzy thresholds, as illustrated in Fig. 3.
• In contrast, the Caselles et al. (1997) method is well suited for structures bounded by strong image intensity edges.

3.2 Software Architecture

• This section describes SNAP functionality and highlights some of the more innovative elements of its user interface architecture.
• SNAP was designed to provide a tight but complete set of features that focus on active contour segmentation.
• It includes tools for viewing and navigating 3D images, manual labeling of regions of interest, combining multiple segmentation results, and post-processing them in 2D and 3D.
• Built on the ITK backbone, SNAP can read and write many image formats, and new features can be added easily.

3.2.1 Image Navigation and Manual Segmentation

• SNAP’s user interface emphasizes the 3D nature of medical images.
• Navigation is aided by a linked 3D cursor whose logical location is at the point where the three orthogonal planes intersect.
• The cursor can be repositioned in each slice view by mouse motion, causing different slices to be shown in the remaining two slice views.
• This approach has the disadvantage that partial volume segmentations can not be represented, but it allows users to view and edit regions simultaneously in the three slice views and in the 3D view.
• Other notable features of SNAP include the image input wizard, which allows users to read a number of recognized image file formats, includes specialized dialogs for DICOM series and raw data, and provides a graphical user interface for specifying the mapping between image and anatomical coordinate systems.

3.2.2 Automatic Segmentation Workflow

• The outcome of active contour segmentation depends on a number of parameters, including the choice of method, the way in which the input image is converted into a probability map or speed function, the initial contour, and the weights assigned to various internal and external forces that drive contour evolution.
• To simplify this task, SNAP organizes parameter specification into a wizard-like workflow and relies extensively on live feedback mechanisms.
• Level set methods allow contours to change topology, and it is common to place several seeds within one structure, letting them merge into a single contour over the course of evolution.
• The user can use ‘stop’, ‘rewind’ and ‘single step’ buttons to control and terminate contour evolution.
• Before entering the automatic segmentation mode, the user may choose to restrict segmentation to a 3D region of interest in order to reduce computational cost and memory use.

4 Results

• The new SNAP tool, with its combination of user-guided 3D active contour segmentation and post-processing via manual tracing in orthogonal slices or using the 3D cut-plane tool, is increasingly replacing conventional 2D slice editing for a variety of image segmentation tasks.
• SNAP is used in several large neuroimaging studies at UNC Chapel Hill, Duke University and the University of Pennsylvania.
• Segmentation either uses the soft threshold option for the definition of foreground and background, e.g. for the segmentation of the caudate nucleus in head MRI, or employs existing tissue probability maps that define object to background probabilities.
• This option is used for the segmentation of ventricles based on cerebrospinal fluid probabilistic segmentations.
• The following sections describe validation of SNAP versus manual rater contour drawing in more details.

4.1 Validation of SNAP: Caudate Segmentation

• From their partnership with the UNC Psychiatry department, the authors have access to a morphologic MRI study with a large set of autistic children (N=56), developmentally delayed subjects (N=11) and control subjects (N=17), scanned at age two.
• SNAP was chosen as an efficient and reliable tool to segment the caudate nucleus from high-resolution MRI.
• Before replacing conventional manual outlining by this new tool, the authors designed a validation study to test the difference between methods, the difference between operators, and the variability for each user.

4.1.1 Gray-level MRI data

• The protocol established by the UNC autism image analysis group rigidly aligns these images to the Talairach coordinate space by specifying anterior and posterior commissure (AC-PC) and the interhemispheric plane.
• The transformation also interpolates the images to the isotropic voxel size of 1mm3.
• Automatic atlas-based tissue segmentation using EMS (Van Leemput et al., 1999a,b) results in a hard segmentation and separate probability maps for white matter, gray matter and cerebrospinal fluid.
• These three-tissue maps are used for SNAP ventricle segmentation, but not for the caudate nucleus, because in some subjects, the intensity distribution of the subcortical gray matter is different from the cortex.

4.1.2 Reliability series and validation

• Five MRI datasets were arbitrarily chosen from the whole set of 100+ images.
• Data were replicated three times and blinded to form a validation database of 15 images.
• Three highly trained raters participated in the validation study; rater A segmented each image manually and in SNAP, while rater B only used SNAP and rater C only performed manual segmentation.
• Segmentation results between pairs of raters or methods were analyzed using common intraclass correlation statistics (ICC) as well as using overlap statistics.

4.1.3 Caudate nucleus segmentation

• At first sight, the caudate seems easy to segment since the largest fraction of its boundary is adjacent to the lateral ventricles and white matter.
• Portions of the caudate boundary can be localized with standard edge detection.
• The caudate is also adjacent to the nucleus accumbens and the putamen where there are no visible boundaries in MRI (see Fig. 10).
• The caudate, nucleus accumbens and putamen are distinguishable on histological slides, but not on T1-weighted MRI of this resolution.
• Another “trouble-spot” for the caudate is where it borders the putamen; there are “fingers” of cell bridges adjacent to blood vessels which span the gap between the two.

4.1.4 Manual boundary drawing

• Using the drawing tools in SNAP, the authors have developed a highly reliable protocol for manual caudate segmentation using slice-by-slice boundary drawing in all three orthogonal views.
• The coupling of cursors between 2D slices and the 3D display help significantly reduce slice-by-slice jitter that is often seen in this type of segmentations.
• Segmentation time for left and right caudate is approximately 1.5 hours for experienced experts.

4.1.5 3D Active Contour Segmentation

• The authors developed a new segmentation protocol for caudate segmentation based on the T1 gray level images with emphasis on efficiency and optimal reliability.
• This results in foreground/background maps which guide the level set evolution.
• In some caudate segmentation protocols, the inferior boundary is cut off by the selection of an axial cut plane, which only takes a few additional seconds using the cut-plane feature in SNAP.
• In their autism project, the authors decided to add a precise separation from the putamen and a masking of the left and right nucleus accumbens.
• This step added another 30 minutes to the whole process.

4.1.6 Volumetric Analysis

• Table 1 lists the left and right caudate volumes for manual segmentation (slice by slice contouring) and user-assisted 3D active contour segmentation (SNAP).
• Results of the reliability analysis using one-way random effects intraclass correlation statistics (Shrout and Fleiss, 1979) are shown in Table 2.
• The table shows not only the excellent reliability of SNAP segmentation but also reflects the excellent reliability of the manual experts trained over several months.
• Therefore, reliability between methods is not significantly different.
• This is to be compared with the significantly reduced segmentation time and short rater training time of SNAP.

4.1.7 Overlap Analysis

• In addition to volume-based reliability analysis, the authors compare SNAP and manual methods in terms of overlap between different segmentations of each instance of the caudate.
• Following the statistical approach described in (Zou et al., 2004), the authors define the overlap between segmentations S1 and S2 as the Dice Similarity Coefficient (DSC): DSC(S1, S1) = 2 Vol(S1 ∩ S2) Vol(S1) + Vol(S2) . (7) This symmetric measure of segmentation agreement lies in the range [0, 1], with larger values indicating greater overlap.
• Table 3 lists means and standard deviations of the overlaps for the left and right caudate within 13 different categories of comparisons.
• Fig. 11 displays box and whisker plots of the same 13 categories.
• Zou et al. (2004) pools the model over all raters, but in their case, since only rater A performed segmentation using both methods, the authors just include segmentations by rater A in the model.

4.2 Lateral Ventricle Segmentation

• Unlike the caudate, which has a simple shape but lacks clearly defined MRI intensity boundaries, the lateral ventricles are complex geometrically yet have an easily identifiable boundary.
• As for the caudate, the authors randomly selected five MRI images from the child autism database and applied their standard image processing pipeline, including tissue class segmentation using EMS.
• The authors ventricle segmentation protocol involves placing three initialization seeds in each ventricle and running the active contour segmentation until there is no more expansion into the horns.
• These problems are corrected by post-processing, which involves reapplying active contour segmentation to the trouble regions or correcting the segmentation manually.
• Overlap statistics are summarized in Table 7 and Fig. 12.

5 Discussion

• The caudate segmentation validation, which compares the SNAP tool to manual segmentation by highly trained raters, demonstrates the excellent reliability of the tool for efficient three-dimensional segmentation.
• In addition to brain structure extraction in MRI, SNAP has found a variety of uses in other imaging modalities and anatomical regions.
• Its automatic segmentation pipeline is limited to a specific subset of segmentation problems where the structure of interest has a different intensity distribution from most of the surrounding tissues.
• These postprocessing tools will be based on graph-theoretic algorithms.

6 Conclusion

• ITK-SNAP is an open source medical image processing application that fulfills a specific and pressing need of biomedical imaging research by providing a combination of manual and semiautomatic tools for extracting structures in 3D image data of different modalities and from different anatomical regions.
• Designed to maximize user efficiency and to provide a smooth learning curve, the user interface is focused entirely on segmentation, parameter selection is simplified using live feedback, and the number of features unrelated to segmentation kept to a minimum.
• Validation in the context of caudate nucleus and lateral ventricle segmentation in child MRI demonstrates excellent reliability and high efficiency of 3D SNAP segmentation, and provides strong motivation for adopting SNAP as the segmentation solution for clinical research in neuroimaging and beyond.

Figures

Table 5: Ventricle volumes (in mm3) from the SNAP reliability experiment. Five test cases, replicated three times (column one) have been segmented by two raters (A, B), who were blinded to the cases.

Fig. 8. User interface for specifying contour evolution parameters, including the relative weights of the forces acting on the contour. The intuitive interface is shown on the left and the mathematical interface on the right. The parameter specification window also shows how the forces interrelate in a two-dimensional example.

Fig. 7. a. User interface for feature image specification: the user is setting the values of the smooth threshold parameters, and the feature image is displayed in the orthogonal slice views using a color map.b. User interface for active contour initialization: the user has placed two spherical bubbles in the caudate nucleus.

Table 6: Intrarater and interrater reliability of lateral ventricle segmentation in SNAP. Reliability was measured based on 3 replications of 5 test datasets by two raters.

Table 2: Intrarater and interrater reliability of caudate segmentation. Reliability was measured based on 3 replications of 5 test datasets by two raters. Reliability values for (1) manual segmentation by two experts; (2) manual vs. SNAP segmentation by the same expert; and (3) SNAP segmentation by two experts show the excellent reliability of both methods and the excellent agreement between manual expert’s segmentation and SNAP. SNAP reduced segmentation time from 1.5 hours to 30 minutes, while the training period to establish reliability was several months for the manual method and significantly shorter for SNAP.

Table 3: Overlap statistics between pairs of caudate segmentations, categorized by different methods and raters. Letters A, B and C refer to individual raters. Overlap values are given in percent, i.e., overlap(S1, S2) = 100%∗DSC(S1, S2). The number of pairs in each category is given in the last column (n).

Fig. 5. a. SNAP user interface shows three orthogonal views of a volumetric image, linked by a common cursor (light blue crosshairs). A fourth panel is used to view the segmented structures in three dimensions. b. Alternatively, SNAP can be focused on a single slice, and zoom facilities are provided for segmenting high resolution images.

Fig. 6. An example of using cut-plane operations to relabel a segmentation into three components (left lateral ventricle, right lateral ventricle and third ventricle.)

Fig. 4. Active contour evolution using the feature image based on region competition. The propagation force acts outwards over the ‘foreground’ region (red) and inwards over the ‘background’ region (blue), causing the active contour to reach equilibrium at the boundary of the regions.

Fig. 3. The plot on the left gives an example of three smooth threshold functions with different values of the smoothness parameter κ. To the right of the plot are the input grayscale image and the feature images corresponding to the three thresholds.

Table 7: Overlap statistics for left and right lateral ventricle segmentations. Overlap values are given as percentages, i.e., overlap(S1, S2) = 100% ∗ DSC(S1, S2).

Fig. 11. Box and whisker plots showing order statistics (minimum, 25% quantile, median, 75% quantile, maximum) of overlaps between pairs of segmentations of the caudate nucleus. The horizontal axis represents 13 different categories of pairwise comparisons that are listed in Table 3, and the vertical axis plots the Dice Similarity Coefficient (DCS). Columns 1-4 are SNAP-to-SNAP comparisons, columns 5-8 are manual-to-manual comparisons, and columns 9-13 are mixed-method comparisons. Orange boxes represent intrarater comparisons for specific raters; red boxes stand for intrarater comparisons pooled over available raters; light blue boxes are interrater comparisons for specific pairs of raters; and dark blue boxes are interrater comparisons pooled over available rater pairs.

Fig. 12. Box and whisker plots of overlaps between pairs of segmentations of the lateral ventricles nucleus. All columns represent SNAP-to-SNAP comparisons; columns 1 and 2 represent intrarater comparisons for specific raters; column 3 plots intrarater comparisons pooled over the two raters; and column 4 shows interrater comparisons.

Table 1: Caudate volumes (in mm3) from the validation study comparing SNAP reliability to manual segmentation. Five test cases, replicated three times (column one) have been segmented by three raters who were blind to the cases.

Fig. 1. An illustration of the parameters κ and λ that determine the shape of the function g that inversely maps the values of image gradient magnitude to the range (0, 1]. The top row shows the shapes of the mapping function under different values of the parameters, and the bottom row shows the resulting feature images.

Fig. 2. Examples of edge-based contour evolution before (top) and after (bottom) adding the advection term. Without advection, the contour leaks past the boundaries of the caudate nucleus because the external force is non-negative.

Table 4: Results of ANOVA experiment to determine whether segmentation repeatability measured in terms of overlap varies according to method (M), rater (R), case (C), anatomical structure (A) or the pair of segmentations involved in the overlap computation (P ).

Fig. 10. Two and three-dimensional views of the caudate nucleus. Coronal slice of the caudate: Original T1-weighted MRI (left), and overlay of segmented structures (middle). Right and left caudate are shown shaded in green and red; left and right putamen are sketched in yellow, laterally exterior to the caudates. The nucleus accumbens is sketched in red outline. Note the lack of contrast at the boundary between the caudate and the nucleus accumbens, and the fine-scale cell bridges between the caudate and the putamen. At right is a 3D view of the caudate and putamen relative to the lateral ventricles.

Fig. 9. The user interface for contour evolution. The image on the left shows SNAP before evolution is run, and the image on the right is taken after the contour runs for a few seconds. In this example, an edge based feature image is used.

Frequently asked questions

Q1. What statistical methods were used to analyze the results of the MRI?

Segmentation results between pairs of raters or methods were analyzed using common intraclass correlation statistics (ICC) as well as using overlap statistics. 

Q2. What is the common use of SNAP in MRI?

In addition to brain structure extraction in MRI, SNAP has found a variety of uses in other imaging modalities and anatomical regions. 

Q3. What is the role of segmentation in neuroimaging?

Segmentation serves as an essential element in a great number of morphometry studies that test various hypotheses about the pathology and pathophysiology of neurological disorders. 

Q4. What is the description of SNAP?

In the field of biomedical image analysis software, SNAP stands out as a fullfeatured tool that is specifically devoted to segmentation. 

Q5. What is the reason why the method is not used in the clinical environment?

Methods based on registration are also very computationally intensive, which may discourage their routine use in the clinical environment. 

Q6. What is the spectrum of available segmentation approaches?

The spectrum of available segmentation approaches is broad, ranging from manual outlining of structures in 2D cross-sections to cutting-edge methods that use deformable registration to find optimal correspondences between 3D images and a labeled atlas (Haller et al., 1997; Goldszal et al., 1998). 

Q7. What is the reason why the authors suspect semiautomatic methods are not being used in clinical research?

The authors suspect that insufficient attention to developing tools that make parameter selection intuitive has prevented semiautomatic methods from replacing manual delineation as the tool of choice in the clinical research environment. 

Q8. What are the two types of forces that are used to define the shape of the contour?

These forces are characterized as internal and external : internal forces are derived from the contour’s geometry, and are used to impose regularity constraints on the shape of the contour, while external forces incorporate information from the image being segmented. 

Q9. What is the common reason why the caudate is easy to segment?

At first sight, the caudate seems easy to segment since the largest fraction of its boundary is adjacent to the lateral ventricles and white matter. 

Q10. What is the way to resample the region of interest?

An option to resample the region of interest using nearest neighbor, linear, cubic, or windowed sinc interpolation is provided; this is recommended for images with anisotropic voxels. 

Q11. What is the protocol used to align the images to the Talairach coordinate space?

The protocol established by the UNC autism image analysis group rigidly aligns these images to the Talairach coordinate space by specifying anterior and posterior commissure (AC-PC) and the interhemispheric plane.