scispace - formally typeset
Open accessPosted ContentDOI: 10.1101/2021.03.01.431313

Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning

02 Mar 2021-bioRxiv (Cold Spring Harbor Laboratory)-
Abstract: Understanding the spatial organization of tissues is of critical importance for both basic and translational research. While recent advances in tissue imaging are opening an exciting new window into the biology of human tissues, interpreting the data that they create is a significant computational challenge. Cell segmentation, the task of uniquely identifying each cell in an image, remains a substantial barrier for tissue imaging, as existing approaches are inaccurate or require a substantial amount of manual curation to yield useful results. Here, we addressed the problem of cell segmentation in tissue imaging data through large-scale data annotation and deep learning. We constructed TissueNet, an image dataset containing >1 million paired whole-cell and nuclear annotations for tissue images from nine organs and six imaging platforms. We created Mesmer, a deep learning-enabled segmentation algorithm trained on TissueNet that performs nuclear and whole-cell segmentation in tissue imaging data. We demonstrated that Mesmer has better speed and accuracy than previous methods, generalizes to the full diversity of tissue types and imaging platforms in TissueNet, and achieves human-level performance for whole-cell segmentation. Mesmer enabled the automated extraction of key cellular features, such as subcellular localization of protein signal, which was challenging with previous approaches. We further showed that Mesmer could be adapted to harness cell lineage information present in highly multiplexed datasets. We used this enhanced version to quantify cell morphology changes during human gestation. All underlying code and models are released with permissive licenses as a community resource.

... read more

Topics: Cell morphology (56%), Segmentation (51%)
Citations
  More

23 results found


Journal ArticleDOI: 10.1016/J.CBPA.2021.04.001
Abstract: A cell's phenotype is the culmination of several cellular processes through a complex network of molecular interactions that ultimately result in a unique morphological signature. Visual cell phenotyping is the characterization and quantification of these observable cellular traits in images. Recently, cellular phenotyping has undergone a massive overhaul in terms of scale, resolution, and throughput, which is attributable to advances across electronic, optical, and chemical technologies for imaging cells. Coupled with the rapid acceleration of deep learning-based computational tools, these advances have opened up new avenues for innovation across a wide variety of high-throughput cell biology applications. Here, we review applications wherein deep learning is powering the recognition, profiling, and prediction of visual phenotypes to answer important biological questions. As the complexity and scale of imaging assays increase, deep learning offers computational solutions to elucidate the details of previously unexplored cellular phenotypes.

... read more

8 Citations


Open accessJournal ArticleDOI: 10.3389/FIMMU.2021.727626
Abstract: Multiplexed imaging is a recently developed and powerful single-cell biology research tool. However, it presents new sources of technical noise that are distinct from other types of single-cell data, necessitating new practices for single-cell multiplexed imaging processing and analysis, particularly regarding cell-type identification. Here we created single-cell multiplexed imaging datasets by performing CODEX on four sections of the human colon (ascending, transverse, descending, and sigmoid) using a panel of 47 oligonucleotide-barcoded antibodies. After cell segmentation, we implemented five different normalization techniques crossed with four unsupervised clustering algorithms, resulting in 20 unique cell-type annotations for the same dataset. We generated two standard annotations: hand-gated cell types and cell types produced by over-clustering with spatial verification. We then compared these annotations at four levels of cell-type granularity. First, increasing cell-type granularity led to decreased labeling accuracy; therefore, subtle phenotype annotations should be avoided at the clustering step. Second, accuracy in cell-type identification varied more with normalization choice than with clustering algorithm. Third, unsupervised clustering better accounted for segmentation noise during cell-type annotation than hand-gating. Fourth, Z-score normalization was generally effective in mitigating the effects of noise from single-cell multiplexed imaging. Variation in cell-type identification will lead to significant differential spatial results such as cellular neighborhood analysis; consequently, we also make recommendations for accurately assigning cell-type labels to CODEX multiplexed imaging.

... read more

2 Citations


Open accessPosted ContentDOI: 10.1101/2021.06.05.447183
Xu Xiao1, Ying Qiao1, Yudi Jiao1, Na Fu1  +4 moreInstitutions (1)
07 Jun 2021-bioRxiv
Abstract: Highly multiplexed imaging technology is a powerful tool to facilitate understanding cells composition and interaction in tumor microenvironment at subcellular resolution, which is crucial for both basic research and clinical applications. Imaging mass cytometry (IMC), a multiplex imaging method recently introduced, can measure up to 40 markers simultaneously in one tissue section by using a high-resolution laser with a mass cytometer. However, due to its high resolution and large number of channels, how to process and interpret the image data from IMC remains a key challenge for its further applications. Accurate and reliable single cell segmentation is the first and a critical step to process IMC image data. Unfortunately, existing segmentation pipelines either produce inaccurate cell segmentation results, or require manual annotation which is very time-consuming. Here, we developed Dice-XMBD, a Deep learnIng-based Cell sEgmentation algorithm for tissue multiplexed imaging data. In comparison with other state-of-the-art cell segmentation methods currently used in IMC, Dice-XMBD generates more accurate single cell masks efficiently on IMC images produced with different nuclear, membrane and cytoplasm markers. All codes and datasets are available at https://github.com/xmuyulab/Dice-XMBD.

... read more

Topics: Segmentation (51%)

1 Citations


Open accessPosted ContentDOI: 10.1101/2021.09.08.459490
Shirley Greenbaum1, Shirley Greenbaum2, Inna Averbukh2, Erin Soon2  +16 moreInstitutions (6)
10 Sep 2021-bioRxiv
Abstract: Beginning in the first trimester, fetally derived extravillous trophoblasts (EVTs) invade the uterus and remodel its spiral arteries, transforming them into large, dilated blood vessels that lack smooth muscle and are partially lined with EVTs instead of vascular endothelium. Several mechanisms have been proposed to explain how EVTs coordinate with decidual cells to promote a tissue microenvironment conducive to spiral artery remodeling (SAR). However, it remains a matter of debate which immune and stromal cell types participate in these interactions, how this process evolves with respect to gestational age, and which anatomic routes are the predominate path of EVT invasion in humans. To elucidate this complex interplay, we used multiplexed ion beam imaging by time of flight with a 37-plex antibody panel to build the first spatio-temporal atlas of the human maternal-fetal interface in the first half of pregnancy at single-cell resolution. We analyzed ~500,000 cells and 588 spiral arteries within intact decidua from 66 patients between 6-20 weeks of gestation. Using custom machine learning algorithms for cell segmentation and classification, we evaluated the spatial distributions and phenotype of 20 maternal and five EVT populations with respect to gestational age and SAR. Gestational age substantially influenced the frequency of most maternal immune and stromal cells, with tolerogenic subsets expressing CD206, CD163, TIM-3, Galectin-9, and IDO-1 preferentially enriched at later time points. In contrast, SAR progression, and not gestational age, preferentially correlated with local invasion of EVTs. Lastly, by comparing spatial co-occurrence and phenotype of decidual interstitial, perivascular and intravascular EVTs with respect to SAR progression, we developed a statistical model suggesting an intravasation mechanism as the predominant route of EVT invasion in superficial decidua. Taken together, these results support a coordinated model of decidualization in which increasing gestational age drives a transition in maternal decidua towards a tolerogenic niche conducive to locally regulated, EVT-dependent SAR.

... read more

Topics: Spiral artery (53%), Decidua (53%), Decidual cells (52%) ... read more

1 Citations


Journal ArticleDOI: 10.1016/J.CCELL.2021.09.008
08 Nov 2021-Cancer Cell
Abstract: Summary Small cell lung cancer (SCLC) is an aggressive malignancy that includes subtypes defined by differential expression of ASCL1, NEUROD1, and POU2F3 (SCLC-A, -N, and -P, respectively). To define the heterogeneity of tumors and their associated microenvironments across subtypes, we sequenced 155,098 transcriptomes from 21 human biospecimens, including 54,523 SCLC transcriptomes. We observe greater tumor diversity in SCLC than lung adenocarcinoma, driven by canonical, intermediate, and admixed subtypes. We discover a PLCG2-high SCLC phenotype with stem-like, pro-metastatic features that recurs across subtypes and predicts worse overall survival. SCLC exhibits greater immune sequestration and less immune infiltration than lung adenocarcinoma, and SCLC-N shows less immune infiltrate and greater T cell dysfunction than SCLC-A. We identify a profibrotic, immunosuppressive monocyte/macrophage population in SCLC tumors that is particularly associated with the recurrent, PLCG2-high subpopulation.

... read more

Topics: Adenocarcinoma (53%), Metastasis (51%)

1 Citations


References
  More

76 results found


Open accessProceedings ArticleDOI: 10.1109/CVPR.2016.90
Kaiming He1, Xiangyu Zhang1, Shaoqing Ren1, Jian Sun1Institutions (1)
27 Jun 2016-
Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

... read more

Topics: Deep learning (53%), Residual (53%), Convolutional neural network (53%) ... read more

93,356 Citations


Open accessJournal ArticleDOI: 10.1038/NMETH.2089
01 Jul 2012-Nature Methods
Abstract: For the past 25 years NIH Image and ImageJ software have been pioneers as open tools for the analysis of scientific images. We discuss the origins, challenges and solutions of these two programs, and how their history can serve to advise and inform other software projects.

... read more

Topics: Image processing (54%), Bioimage informatics (52%)

34,063 Citations


Open accessJournal ArticleDOI: 10.1038/NMETH.2019
01 Jul 2012-Nature Methods
Abstract: Fiji is a distribution of the popular open-source software ImageJ focused on biological-image analysis. Fiji uses modern software engineering practices to combine powerful software libraries with a broad range of scripting languages to enable rapid prototyping of image-processing algorithms. Fiji facilitates the transformation of new algorithms into ImageJ plugins that can be shared with end users through an integrated update system. We propose Fiji as a platform for productive collaboration between computer science and biology research communities.

... read more

Topics: Software design (51%), Software (50%)

30,888 Citations


Open accessPosted Content
02 Jan 2012-arXiv: Learning
Abstract: Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from this http URL.

... read more

28,898 Citations


Open accessBook ChapterDOI: 10.1007/978-3-319-24574-4_28
05 Oct 2015-
Abstract: There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .

... read more

Topics: Brain segmentation (57%), Deep learning (54%), Segmentation (50%)

28,273 Citations


Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
202123