High-accuracy neurite reconstruction for high-throughput neuroanatomy

TL;DR: A method for fast and reliable reconstruction of densely labeled data sets, based on manually skeletonizing each neurite redundantly with a visualization-annotation software tool called KNOSSOS, is developed, which is ∼50-fold faster than volume labeling.

Abstract: This Technical Report describes an automated algorithm to trace densely labeled neurons and reconstruct their structure, thus providing a new tool in functional connectome analysis.

Summary

INTRODUCTION

• Almost all available neuroanatomical data at single-cell resolution stem from such experiments, but as fluorescence imaging data from samples with a much higher staining density are becoming available (hundreds of neurons per 1 mm3, labeled using various genetic or virus-based techniques6-7), high reconstruction reliability can no longer be presumed.
• Extracting information about neuron morphology and circuit structure from such data poses two major challenges.
• Some of these decisions are difficult and, more importantly, because they have to be made constantly while annotating, their reliability depends on the uninterrupted attentiveness of the human annotator.
• The authors quantified discrepancies between multiple skeletons of the same neurite and, based on their distribution, optimized the correction of errors and the creation of a consensus skeleton (which is actually a bundle of closely spaced skeleton pieces).

Browsing large-scale EM data

• The authors first developed a software tool (KNOSSOS, s. Supplementary Movie) for browsing and annotating large-scale volume data.
• KNOSSOS allows quick navigation along all axes by selectively loading only the data surrounding the currently viewed location.
• This allowed us to distribute the work load to a large number of non-expert annotators (in their case >80 undergraduate students).
• Then, the user advances through the data along a neurite, and places nodes at intervals of approximately 7-10 image 7 planes, approximately at the center of the neurite.
• Skeletonization allows the user to focus annotation to the core line of a neurite.

Discrepancies between skeletons

• The authors next investigated how frequently annotators disagreed when skeletonizing the same neurite, starting from the same initial location.
• To detect errors in the skeletons, the authors asked multiple annotators to skeletonize the same neurite (Fig. 3a).
• The authors then counted the number of edges that had a certain combination of agreeing and total votes (say, 6 agreeing votes out of 10 total votes), and reported these for all encountered combinations of agreeing and total votes in a 2-dimensional vote histogram (Fig. 3f).
• The authors found complete agreement between annotators (number of agreeing votes equal to the total number of votes, evaluated for edges with at least three votes) for 68 % of all locations, for 8 % only one annotator disagreed, and 10 % of the locations were annotated by only one annotator.
• We, therefore, based their consensus rule for an edge on whether the estimated distribution of edge detectability given the agreeing and disagreeing votes cast for that edge, p(pe|(T,N)), indicated that the edge at that location was more likely to be detected than not.

Annotator quality

• So far the authors have assumed that the error rates of different annotators are similar.
• To calculate the error probabilities for eliminated edges and accepted edges the authors integrated the distributions of edge detectability given the agreeing and disagreeing votes cast for that edge, p(pe|(T,N)), for pe>0.5, and pe<0.5, respectively (Fig. 4a).
• 15 As the number of annotators rises the accuracy of the consensus skeleton increases (Fig. 5c) initially steeply but then more slowly.
• Since for most locations connectedness is easy to determine, increasing the overall redundancy is wasteful.
• In order to determine the redundancy-accuracy tradeoff for focused re-annotation the authors performed Monte-Carlo simulations and found that for focused re-annotation the accuracy should rise much more steeply, almost exponentially, with the average redundancy (Fig. 5c).

Dense reconstruction

• To illustrate the feasibility of dense neuron reconstruction from SBEM data using the tools presented here, the authors selected all rod bipolar cells (RBCs, Fig. 6) from a SBEM data set that is in the process of being skeletonized (data set E2006, currently at 2 fold redundancy, Helmstaedter et al., in preparation).
• RBCs were initially identified on the basis of geometrical parameters using automatic clustering (Helmstaedter et al., in preparation).
• The remaining 114 cells displayed the tiling patterns of axons and dendrites expected for rod bipolar cells (Fig. 6c,d).
• The annotation speed for these skeletons was 5.3 h per mm path length (the RBCs had an average neurite length of 368±103 µm, mean±s.d.).
• Using the model described above, the authors expect about 10 errors per cell for double annotation.

Dense vs. sparse reconstruction

• The authors data show that the dense reconstruction of neurites in SBEM volume electron microscopy data is feasible, but also that manual annotations contain errors, even when performed by experts.
• While the identification of synapses can be error-prone as well, one such error affects only one particular synapse, with a much less severe effect on the connectomic reconstruction error than the typical neurite continuity error has.
• Mass annotation, distribution of skill and training levels Finding the consensus of multiple annotations using RESCOP may reduce the error rate to a level sufficient for almost any application of connectomic circuit reconstruction.
• The low density of difficult locations also means that ambiguous vote ratios (T/N near 0.5) are rare and the fits for p(pe) are not very well constrained in the region around pe=0.5, making estimates of error rates for large N somewhat uncertain (Supplementary Fig. 5).
• One advantage of using weakly trained annotators is that the reliability increase can be achieved at a lower cost than with expert proof readers, who might still make attentionrelated errors at an unacceptable rate (Fig. 2); Also, requiring PhD-students or post-docs to do several thousand hours of annotating is hardly a good use of their talents.

Author contributions:

• The procedure used to measure the agreement between multiple annotators, shown schematically for one skeleton edge (dashed line) in skeleton A. (f) Histograms of edge votes for the 50-fold annotation of one cell (left panel) and the dense skeletonization of 98 neurites (right panel).
• The probabilities for different T (number of pro votes) for one edge (i, binomial distribution for pe=0.7 and N=10 annotators); and for all edges combined (k, schematic), also known as Bottom panels.
• Same analysis for a conventionally stained dataset annotated using the original data (blue), with added noise , and at half the resolution (cyan). (c, d) view onto the plane of the retina confined (as indicated in a) to the dendrites (c) and axons (d) of the bipolar cells, respectively, also known as Bottom panel.

SBEM

• For E1088 the imaged region spanned the inner plexiform layer of the retina and included parts of the inner nuclear and of the ganglion cell layers.
• After edge elimination, the authors collected all skeleton nodes for all redundantly annotated skeletons that still were connected to a source seed area near the soma by a continuous path of edges (using connected components).
• The calculation made to decide whether or not to eliminate an edge can be extended to calculate the probability that the decision was wrong and that the RESCOPed consensus skeleton therefore contains an error at that point.
• The set accuracy goal was then corrected for the residual errors for those runs that reached Nmax (Nmax = 6000, e1088 single-cell data and k0563 data, Fig. 5c, Supplementary Fig. 5d), with the exception of the dense skeletonization data where the number of runs that reached Nmax was small.

Figures

Figure 1 Comparison of volume and skeleton annotation. Examples of volume labeling (a) and skeletonization (b) for the same two neurite fragment; cell-surface labeled data (dataset E1088, s. Methods). (c) Sketch of a neurite skeleton. (d) Rate of time consumption for volume labeling10 and for skeleton annotation (data from this study; annotated using KNOSSOS, s. Supplementary Movie), for both cell-surface labeled data (black) and the conventionally stained dataset (K0563, gray, s. Fig. 5d). Error bars: range (volume labeling), s.d. (skeletoniziation). (e). Outline of the Redundant Skeleton COnsensus Procedure (RESCOP) as described in this article. Scale bar, 250 nm (a,b).

Full text

HAL Id: hal-00658165
https://hal.archives-ouvertes.fr/hal-00658165
Submitted on 10 Jan 2012
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
High-accuracy neurite reconstruction for
high-throughput neuroanatomy
Moritz Helmstaedter, Kevin L Briggman, Winfried Denk
To cite this version:
Moritz Helmstaedter, Kevin L Briggman, Winfried Denk. High-accuracy neurite reconstruc-
tion for high-throughput neuroanatomy. Nature Neuroscience, Nature Publishing Group, 2011,
�10.1038/nn.2868�. �hal-00658165�

High-accuracy neurite reconstruction for high-throughput
neuroanatomy.
Moritz Helmstaedter, Kevin L Briggman, and Winfried Denk
Max Planck Institute for Medical Research, Jahnstr. 29, D-69120 Heidelberg, Germany.
Running title: High-accuracy 3D EM skeletonization
Key words: serial block-face electron microscopy, neural circuit reconstruction,
connectomics
Editorial correspondence: Moritz Helmstaedter, Max Planck Institute for Medical
Research, Jahnstr. 29, D-69120 Heidelberg, Germany. Phone: +49 6221 486 149. Fax: +49
6221 486 325; E-mail: moritz.helmstaedter@mpimf-heidelberg.mpg.de
Competing financial interests: Published Patent Application US 20100183217 (MH and
WD). IP License income from Gatan Inc. for Serial Blockface Imaging (WD).

2
ABSTRACT
Neuroanatomic analysis depends on the reconstruction of complete cell shapes. High-
throughput reconstruction of neural circuits (“connectomics”) using volume electron
microscopy requires dense staining of all cells, where even experts make annotation errors.
Currently, reconstruction rather than acquisition speed limits the determination of neural
wiring diagrams. We present methods for the fast and reliable reconstruction of densely
labeled datasets. Our approach, based on manually skeletonizing each neurite redundantly
(multiple times) with a special visualization/annotation software tool (KNOSSOS), is ~50
times faster than volume labeling. Errors are detected and eliminated by a “redundant-
skeleton consensus procedure” (RESCOP), which uses a statistical model of how true
neurite connectivity is transformed into annotation decisions. RESCOP also estimates the
consensus skeletons’ reliability. Focused re-annotation of difficult locations promises a
rather steep increase of reliability as a function of the average skeleton redundancy and
thus the nearly error-free analysis of large neuroanatomical datasets.

3
INTRODUCTION
The reconstruction of neuronal circuits has been a central approach toward
understanding the function of the nervous system since the earliest studies by Golgi and
Ramòn y Cajal
1-2
. While many neurons extend over tens of centimeters, the caliber of thin
neurites can be as small as 40 nm (spine necks
3
). This range of length scales is bound to
challenge any method aimed at the extraction of neuron morphology from the data. For
sparsely stained tissue, with only a small fraction of all neurons labeled, such as with the
Golgi method
2
or by selective dye injection
4-5
, imaging techniques operating at a resolution
of around 1 µm are sufficient to follow all processes. This holds true even if the neurite
caliber falls well below the imaging resolution, because in very sparsely stained data the
identity of each neurite is easily established. Manual reconstructions of individual neurons
from such data are, therefore, assumed to be highly reliable, even though little validation of
this reliability has been reported. Almost all available neuroanatomical data at single-cell
resolution stem from such experiments, but as fluorescence imaging data from samples
with a much higher staining density are becoming available (hundreds of neurons per 1
mm
3
, labeled using various genetic or virus-based techniques
6-7
), high reconstruction
reliability can no longer be presumed.
For the reconstruction of complete cellular wiring diagrams (“connectomes”
8-9
) assuring
reconstruction reliability is even more difficult because the morphologies of all neurons,
not only those of a small subset, have to be extracted. This may eventually be possible at
light-microscopic resolution by staining all neurons with a sufficient number of
distinguishable colors
7, 9
but otherwise requires imaging at a resolution high enough to
follow all neurites in densely packed neuropil (discussed in
10
). Such a reconstruction was
performed for the entire nervous system (302 neurons) of the nematode C. elegans
11
using
serial-section electron microscopy.

4
Recently developed techniques for automated volume electron microscopy
12-15
enable
the imaging of volumes large enough to contain more complex neural circuits
16
. However,
extracting information about neuron morphology and circuit structure from such data poses
two major challenges. First, the total neurite path length in many neural circuits is typically
in the range of meters (at least 0.3 m for small circuits such as a (100 µm)
3
region of retina,
and as much as 400 m for a mouse cortical column
10
). Using currently available software
tools for neurite contouring (e.g., Reconstruct
17
) the complete analysis of such circuits is
very slow and thus prohibitively expensive. Contouring every neurite for a path length of
0.3 m would require an estimated 60,000 hours (30 person years) of annotation time.
Reconstruction accuracy is the second major concern. While for sparsely stained data the
selectivity of the stain makes following the neurites easy, connectomic reconstruction
requires a large number of decisions (as many as one every ~4 µm in the retina) about
whether to continue, branch, or terminate a neurite. Some of these decisions are difficult
and, more importantly, because they have to be made constantly while annotating, their
reliability depends on the uninterrupted attentiveness of the human annotator. As a third
obstacle, synapses must be identified with sufficient accuracy.
Here, we describe a set of tools that substantially improve both the speed and the
accuracy of neurite reconstruction. We chose to annotate the data by following a single
core line along the inside of each neurite, creating a “skeleton” representation of each
neuron’s morphology. When using the KNOSSOS software tool, which we developed for
the convenient browsing and annotation of large datasets, we observed a 50-fold (range 20-
130-fold) increase in the amount of neurite path length reconstructed per unit time. We
quantified discrepancies between multiple (redundant) skeletons of the same neurite and,
based on their distribution, optimized the correction of errors and the creation of a
consensus skeleton (which is actually a bundle of closely spaced skeleton pieces). We call
our method “REdundant-Skeleton COnsensus Procedure, RESCOP”, with ‘redundant’ used

Frequently asked questions

Q1. What are the contributions in "High-accuracy neurite reconstruction for high-throughput neuroanatomy" ?

The authors present methods for the fast and reliable reconstruction of densely labeled datasets. Their approach, based on manually skeletonizing each neurite redundantly ( multiple times ) with a special visualization/annotation software tool ( KNOSSOS ), is ~50 times faster than volume labeling. 

Q2. How many millimeters of neurite need to be followed?

In order to densely reconstruct even a local neuronal circuit, at least several hundred millimeters of neurite need to be correctly followed. 

Q3. What software was used to annotate the skeletons of the retina?

Reconstruction softwareNeurite skeletons were annotated using KNOSSOS (written in C by Jörgen Kornfeld and Fabian Svara according to specifications by the authors). 

Q4. What is the way to determine the distribution of edge detectabilities p(pe)?

In fact, a delta function at pe=0 can be added to the distribution of edge detectabilities p(pe) without changing the goodness of the fit and without affecting the following results. 

Q5. How many decisions do you have to make to reconstruct a neurite?

While for sparsely stained data the selectivity of the stain makes following the neurites easy, connectomic reconstruction requires a large number of decisions (as many as one every ~4 µm in the retina) about whether to continue, branch, or terminate a neurite. 

Q6. Why is it necessary to determine which skeleton pieces still belong together?

Because edge elimination splits some skeletons (Fig. 4c), it is necessary to determine which skeleton pieces still belong together. 

Q7. What is the way to measure the edge detectability?

The edge detectability depends on whether the points are actually connected (see below), but it also varies as a consequence of the local neurite geometry (wide, straight, or bundled neurites are, for example, easier to follow) and local staining quality. 

Q8. What is the advantage of using weakly trained annotators?

One advantage of using weakly trained annotators is that the reliability increase can be achieved at a lower cost than with expert proof readers, who might still make attentionrelated errors at an unacceptable rate (Fig. 2); Also, requiring PhD-students or post-docs to do several thousand hours of annotating is hardly a good use of their talents. 

Q9. Why does the error rate decrease as the number of votes increases?

Because the distribution of edge detectability given the votes p(pe|(T,N)) becomes more sharply peaked as the total number of votes increases (Fig. 5b), the error rate for a given ratio of agreeing to total votes decreases. 

Q10. What is the average number of disagreements between the skeletons?

The authors found the average number of disagreements to be 1.0±0.4, 2.1±0.3, 7.2±0.9, and 15.5±3.5 (mean ± s.e.m.) for the 25-fold, 10-fold, 5-fold and single skeletons respectively, corresponding to mean distances between errors of 600.2 µm, 281.3 µm, 83.4 µm and 38.7 µm (Fig 5c, top panel). 

Q11. How many errors are there in the annotation of a solid body of EM data?

For the study of local synaptic geometry, where a solid body of serial EM studies exists, a modest error rate will only rarely affect the conclusions.