Quantitative evaluation of 10 tractography algorithms on a realistic diffusion MR phantom.

TLDR: A common dataset with known ground truth and a reproducible methodology to quantitatively evaluate the performance of various diffusion models and tractography algorithms is used and evidence that diffusion models such as (fiber) orientation distribution functions correctly model the underlying fiber distribution is provided.

About: This article is published in NeuroImage.The article was published on 2011-05-01 and is currently open access. It has received 410 citations till now. The article focuses on the topics: Diffusion MRI & Tractography.

Figures

Figure 3: ADC and FA images of the 6x6x6 phantom dataset. Top: The single slice of the ADC image is shown for the three b-values used for acquisition. The values reported are the mean (standard deviation) of the ADC values of the phantom. Bottom: The single slice of the FA image is shown for the three b-values used for acquisition. The values reported are the mean (standard deviation) of the FA values of the phantom.

Figure 10: Examples of tractography on a real dataset of three methods. For each method, the entire white matter was reconstructed and only fibers passing trough the ROIs shown on Figure 9 are shown. Left: Fibers passing through the corpus callosum (Fig. 9 left). Right: Fibers passing through the pons (Fig. 9 right). Results are shown for: top: single-DT and streamline tractography (method 4), middle: FOD with constrained spherical deconvolution and streamline tractography (2nd best method), and bottom: for global tractography (winning method). The color codes the main fiber orientation: a left-right oriented fiber is colored in red, an infero-superior fiber is blue and an antero-posterior fiber is green.

Figure 8: Probabilistic tractography of the phantom dataset using FSL. The connectivity maps are shown on top of the b0 image. We refer the reader to Figure 4 for the seed locations. The color range goes from yellow (low probability of connexion) to red (high probability). The obtained CMs indicate that probabilistic tractography remains feasible on such synthetic dataset, which could be further used to evaluate this class of algorithm.

Figure 7: Quantitative comparison of the 10 contributions to the ground truth using the spatial metric of Eq. 3 (left), the tangent metric of Eq. 4 (middle), and the curve metric of Eq. 5 (right). Top: The metric scores for each method and each fiber. Units are in mm for the spatial metric, in degree for the tangent metric and in mm −1 for the curve metric. Bottom: Points attributed to each method. Color-coding is the following: dark blue: 0 point, light blue: 1 point, yellow: 2 points, and brown: 3 points. The total number of points for each method and each metric is indicated below the x-axis.

Table 3: Seed-dependent parameters for method 8.

Figure 2: ADC and FA images of the 3x3x3 phantom dataset. Top: The middle slice of the ADC image is shown for the three b-values used for acquisition. The values reported are the mean (standard deviation) of the ADC values of the phantom. Bottom: The middle slice of the FA image is shown for the three b-values used for acquisition. The values reported are the mean (standard deviation) of the FA values of the phantom.

Frequently asked questions

Q1. What are the contributions mentioned in the paper "Quantitative evaluation of 10 tractography algorithms on a realistic diffusion mr phantom" ?

In this work, the authors use a common dataset with known ground truth and a reproducible methodology to quantitatively evaluate the performance of various diffusion models and tractography algorithms. The results provide evidence that: 1. For high SNR datasets, diffusion models such as ( fiber ) orientation distribution functions correctly model the underlying fiber distribution and can be used in conjunction with streamline tractography, and 2. 

Q2. What are the future works mentioned in the paper "Quantitative evaluation of 10 tractography algorithms on a realistic diffusion mr phantom" ?

In the future, new evaluation criteria will be proposed. Another possibility is to evaluate whether the boundaries of a bundle are correctly reconstructed by measuring the spatial distance in-between two tracts delimiting the bundle. The authors believe that such a common dataset along with the methodology proposed here can serve as an evaluation basis for existing and new algorithms. New results can be submitted for evaluation by emailing them to fibercup09 @ gmail. 

Q3. What was the role of the diffusion model in motivating the project?

The potential of tractography to help map anatomical connections played a significant role in motivating an ambitious project to map the human ”connectome” 1. 

Q4. Why did the authors choose to use 16 spatial positions?

The choice of those 16 spatial positions was made to ensure that a single fiber bundle passes through each of them to avoid ambiguity on the result and to facilitate the evaluation. 

Q5. What are the popular tractography algorithms?

Among deterministic tractography algorithms, streamline algorithms were developed first [Mori et al., 1999b,Conturo et al., 1999,Basser et al., 2000], followed by more elaborated tensor deflection algorithms [Weinstein et al., 1999,Lazar et al., 2003] or more global approaches [Poupon et al., 2001, Mangin et al., 2002]. 

Q6. How many fibers were used to fill the phantom?

the positive and negative prints were squeezed while keeping fibers strongly tightened until the openings (i.e, where the fibers enter/leave the phantom) are exactly 1cm thick. 

Q7. How many fibers are captured in the phantom?

Compression is carefully controlled to make sure that fibers are captured in 1mm2 crosssection everywhere throughout the phantom. 

Q8. Why did the authors exclude probabilistic tractography from the evaluation panel?

the nature of the ground truth itself prevents the inclusion of probabilistic tractography algorithms into the evaluation panel, since those output gener-9ally connectivity maps (CM) and not fiber pathways. 

Q9. What are the popular probabilistic tractography methods?

Probabilistic tractography methods include DT-based algorithms [Parker et al., 2003,Behrens et al., 2003,Lazar and Alexander, 2005,Friman et al., 2006,RamirezManzanares and Rivera, 2006,Savadjiev et al., 2008,Koch et al., 2002,Zhang et al., 2009], calculation of geodesics in a DT-warped space [Lenglet, 2006, Jbabdi et al., 2004], and numerous HARDI-based methods [Parker and Alexander, 2005, Perrin et al., 2005a,Seunarine et al., 2006,Behrens et al., 2007b,Jbabdi et al., 2007,Savadjiev et al., 2008, Chao et al., 2007a, Seunarine et al., 2007, Haroon and Parker, 2007,Kaden et al., 2007,Jeurissen et al., 2010]. 

Q10. How many fibers are found in the phantom?

Since the authors know the number of fibers and the space they are captured in, the authors can deduce the density of fibers, which was close to 1900 fibers/mm2 everywhere, including in the crossings. 

Q11. What are the objectives of this study?

The objectives of this study are to provide a qualitative and quantitative comparison of several tractography methods on the same realistic dataset with known ground truth and to freely distribute this dataset along with the evaluation methodology so that new methods can be easily evaluated and compared to existing ones. 

Q12. What is the procedure for comparing two consecutive fibers?

This procedure ensures that the function c is monotonically increasing, i.e., if s1 >= s2, c(s1) >= c(s2), which13ensures that two consecutive points of a fiber are associated to two other consecutive points.