Nuclear segmentation in digital microscopic tissue images can enable extraction of high-quality features for nuclear morphometrics and other analysis in computational pathology. Conventional image processing techniques, such as Otsu thresholding and watershed segmentation, do not work effectively on challenging cases, such as chromatin-sparse and crowded nuclei. In contrast, machine learning-based segmentation can generalize across various nuclear appearances. However, training machine learning algorithms requires data sets of images, in which a vast number of nuclei have been annotated. Publicly accessible and annotated data sets, along with widely agreed upon metrics to compare techniques, have catalyzed tremendous innovation and progress on other image classification problems, particularly in object recognition. Inspired by their success, we introduce a large publicly accessible data set of hematoxylin and eosin (H&E)-stained tissue images with more than 21000 painstakingly annotated nuclear boundaries, whose quality was validated by a medical doctor. Because our data set is taken from multiple hospitals and includes a diversity of nuclear appearances from several patients, disease states, and organs, techniques trained on it are likely to generalize well and work right out-of-the-box on other H&E-stained images. We also propose a new metric to evaluate nuclear segmentation results that penalizes object- and pixel-level errors in a unified manner, unlike previous metrics that penalize only one type of error. We also propose a segmentation technique based on deep learning that lays a special emphasis on identifying the nuclear boundaries, including those between the touching or overlapping nuclei, and works well on a diverse set of test images.

A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology

Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images

Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.

Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review

Cell Population Tracking and Lineage Construction with Spatiotemporal Context

It is important to observe and study cancer cells' cycle progression in order to better understand drug effects on cancer cells. Time-lapse microscopy imaging serves as an important method to measure the cycle progression of individual cells in a large population. Since manual analysis is unreasonably time consuming for the large volumes of time-lapse image data, automated image analysis is proposed. Existing approaches dealing with time-lapse image data are rather limited and often give inaccurate analysis results, especially in segmenting and tracking individual cells in a cell population. In this paper, we present a new approach to segment and track cell nuclei in time-lapse fluorescence image sequence. First, we propose a novel marker-controlled watershed based on mathematical morphology, which can effectively segment clustered cells with less oversegmentation. To further segment undersegmented cells or to merge oversegmented cells, context information among neighboring frames is employed, which is proved to be an effective strategy. Then, we design a tracking method based on modified mean shift algorithm, in which several kernels with adaptive scale, shape, and direction are designed. Finally, we combine mean-shift and Kalman filter to achieve a more robust cell nuclei tracking method than existing ones. Experimental results show that our method can obtain 98.8% segmentation accuracy, 97.4% cell division tracking accuracy, and 97.6% cell tracking accuracy

Nuclei Segmentation Using Marker-Controlled Watershed, Tracking Using Mean-Shift, and Kalman Filter in Time-Lapse Microscopy

In this paper, we present a new method combining watershed and mean shift for segmentation and tracking of cancer cell nuclei in time-lapse fluorescence. First, we apply the watershed algorithm to segment the cells in each frame of the video sequence, including clustered cells. Second, mean shift method is employed to track each cell in its cycle progression. The proposed method can automatically segment and track all cells without any manual initialization. Experimental result shows that our method can detect almost all the touching cells and track them successfully, especially in the case of cell mitosis which is a difficult task using traditional methods such as snake and level set.

Automated segmentation and tracking of cells in time-lapse microscopy using watershed and mean shift

Color demosaicing is an essential step in the camera image processing pipeline, especially on hardware platforms for real-time video applications. Though many demosaicing algorithms have been proposed in the past two decades, there remains a substantial gap between industrial needs and academic research. On one hand, industry requires a high perceptual quality, artifact-free, and low-cost demosaicing algorithm that is noniterative and small window based ready to be implemented on a hardware platform with a limited line buffer. On the other hand, academia is targeting high PSNR/SSIM, with the computation cost and line buffer receiving second priority, and often a frame buffer and iterative operation are used. The cost, large line buffer and iteration requirement make most existing demosaicing algorithms inapplicable on hardware platforms. In this paper, we introduce a novel low-cost demosaicing algorithm to narrow the gap. We keep the operation window size and computation cost as the first priority and achieve both high PSNR/SSIM and visual perceptual quality compared to previous state-of-the-art methods on standard test datasets. We fully investigate the a priori knowledge of natural scene raw images and find several key cues that are beneficial to demosaicing. Our demosaicing algorithm is a smart fusion of these useful cues. Furthermore, we solve typical demosaicing issues that occur in many traditional methods, including false color artifacts, T-section closing, and zippering artifacts. The proposed method has no learning stage, no iteration operation, a small line buffer and a limited number of parameters, so it can easily be applied to hardware platforms.

MCFD: A Hardware-Efficient Noniterative Multicue Fusion Demosaicing Algorithm

In this paper, we aim to address the cell phase identification problem, and two important aspects, the feature extraction methods and the classifier design, are discussed. In our study, we first propose extracting high frequency information of different orientations using Steerable filters. Next, we employ a multi-layer neural network using the back-propagation algorithm to replace K-Nearest Neighbor (KNN) classifier which has been implemented in the Cellular Image Quantitator (CELLIQ) system [3]. Experimental results provide a comparison between the proposed steerable filter features and existing regular features which have been used in published papers [3, 5]. From the comparison, it can be concluded that Steerable filter features can effectively represent the cells in different phases and improve the classification accuracy. Neural network also has a better performance than KNN currently deployed in CELLIQ system [3].

Identification of cell-cycle phases using neural network and steerable filter features

Demosaicing and denoising are two important procedures in image signal processing. Although they can be performed separately, the better approach is to perform demosaicing and denoising jointly. From traditional methods to deep learning methods, many joint demosaicing and denoising algorithms are available, but few of them can achieve good image quality while maintaining a low cost. In this paper, we propose a new high-quality, low-cost, and hardware-oriented joint demosaicing and denoising algorithm. First, we introduce an efficient interpolation-based demosaicing algorithm. Then, we develop a new edge-preserving wavelet denoising algorithm with a novel voting-based direction estimation method. Finally, we present our innovative joint demosaicing and denoising algorithm. Our method conducts demosaicing and denoising alternatively, sharing the same direction information. The obtained experimental results show that the new algorithm can achieve better performance with less than 1/40 of the processing time required by other traditional algorithms.

Xiaodong Yang

Papers

Nuclei Segmentation Using Marker-Controlled Watershed, Tracking Using Mean-Shift, and Kalman Filter in Time-Lapse Microscopy

Automated segmentation and tracking of cells in time-lapse microscopy using watershed and mean shift

MCFD: A Hardware-Efficient Noniterative Multicue Fusion Demosaicing Algorithm

Identification of cell-cycle phases using neural network and steerable filter features

Hardware-Oriented Shallow Joint Demosaicing and Denoising