Showing papers by "Jiang Liu published in 2021"
TL;DR: Wang et al. as discussed by the authors proposed a self-attention mechanism in the encoder and decoder to learn rich hierarchical representations of curvilinear structures and employed a 1×3 and a 3×1 convolutional kernel to capture boundary features.
99 citations
TL;DR: In this article, a split-based coarse-to-fine vessel segmentation network for OCTA images (OCTA-Net) was proposed, with the ability to detect thick and thin vessels separately.
Abstract: Optical Coherence Tomography Angiography (OCTA) is a non-invasive imaging technique that has been increasingly used to image the retinal vasculature at capillary level resolution. However, automated segmentation of retinal vessels in OCTA has been under-studied due to various challenges such as low capillary visibility and high vessel complexity, despite its significance in understanding many vision-related diseases. In addition, there is no publicly available OCTA dataset with manually graded vessels for training and validation of segmentation algorithms. To address these issues, for the first time in the field of retinal image analysis we construct a dedicated Retinal OCTA SEgmentation dataset (ROSE), which consists of 229 OCTA images with vessel annotations at either centerline-level or pixel level. This dataset with the source code has been released for public access to assist researchers in the community in undertaking research in related topics. Secondly, we introduce a novel split-based coarse-to-fine vessel segmentation network for OCTA images (OCTA-Net), with the ability to detect thick and thin vessels separately. In the OCTA-Net, a split-based coarse segmentation module is first utilized to produce a preliminary confidence map of vessels, and a split-based refined segmentation module is then used to optimize the shape/contour of the retinal microvasculature. We perform a thorough evaluation of the state-of-the-art vessel segmentation models and our OCTA-Net on the constructed ROSE dataset. The experimental results demonstrate that our OCTA-Net yields better vessel segmentation performance in OCTA than both traditional and other deep learning methods. In addition, we provide a fractal dimension analysis on the segmented microvasculature, and the statistical analysis demonstrates significant differences between the healthy control and Alzheimer’s Disease group. This consolidates that the analysis of retinal microvasculature may offer a new scheme to study various neurodegenerative diseases.
94 citations
TL;DR: Inspired by CycleGAN based on the global constraints of the adversarial loss and cycle consistency, the proposed CSI-GAN treats low and high quality images as those in two domains and computes local structure and illumination constraints for learning both overall characteristics and local details.
Abstract: The development of medical imaging techniques has greatly supported clinical decision making. However, poor imaging quality, such as non-uniform illumination or imbalanced intensity, brings challenges for automated screening, analysis and diagnosis of diseases. Previously, bi-directional GANs (e.g., CycleGAN), have been proposed to improve the quality of input images without the requirement of paired images. However, these methods focus on global appearance, without imposing constraints on structure or illumination, which are essential features for medical image interpretation. In this paper, we propose a novel and versatile bi-directional GAN, named Structure and illumination constrained GAN (StillGAN), for medical image quality enhancement. Our StillGAN treats low- and high-quality images as two distinct domains, and introduces local structure and illumination constraints for learning both overall characteristics and local details. Extensive experiments on three medical image datasets (e.g., corneal confocal microscopy, retinal color fundus and endoscopy images) demonstrate that our method performs better than both conventional methods and other deep learning-based methods. In addition, we have investigated the impact of the proposed method on different medical image analysis and clinical tasks such as nerve segmentation, tortuosity grading, fovea localization and disease classification.
50 citations
27 Sep 2021
TL;DR: Wang et al. as discussed by the authors proposed a multi-branch hybrid transformer-body-edge network (MBT-Net) based on the transformer and body-edge branches for corneal endothelial cell segmentation.
Abstract: Corneal endothelial cell segmentation plays a vital role in quantifying clinical indicators such as cell density, coefficient of variation, and hexagonality. However, the corneal endothelium’s uneven reflection and the subject’s tremor and movement cause blurred cell edges in the image, which is difficult to segment, and need more details and context information to release this problem. Due to the limited receptive field of local convolution and continuous downsampling, the existing deep learning segmentation methods cannot make full use of global context and miss many details. This paper proposes a Multi-Branch hybrid Transformer Network (MBT-Net) based on the transformer and body-edge branch. Firstly, we use the convolutional block to focus on local texture feature extraction and establish long-range dependencies over space, channel, and layer by the transformer and residual connection. Besides, we use the body-edge branch to promote local consistency and to provide edge position information. On the self-collected dataset TM-EM3000 and public Alisarine dataset, compared with other State-Of-The-Art (SOTA) methods, the proposed method achieves an improvement.
29 citations
TL;DR: In this paper, a structure-texture correspondence memory (STCM) module is proposed to reconstruct image texture from its structure, where a memory mechanism is used to characterize the mapping from the normal structure to its corresponding normal texture.
Abstract: This work focuses on image anomaly detection by leveraging only normal images in the training phase. Most previous methods tackle anomaly detection by reconstructing the input images with an autoencoder (AE)-based model, and an underlying assumption is that the reconstruction errors for the normal images are small, and those for the abnormal images are large. However, these AE-based methods, sometimes, even reconstruct the anomalies well; consequently, they are less sensitive to anomalies. To conquer this issue, we propose to reconstruct the image by leveraging the structure-texture correspondence. Specifically, we observe that, usually, for normal images, the texture can be inferred from its corresponding structure (e.g., the blood vessels in the fundus image and the structured anatomy in optical coherence tomography image), while it is hard to infer the texture from a destroyed structure for the abnormal images. Therefore, a structure-texture correspondence memory (STCM) module is proposed to reconstruct image texture from its structure, where a memory mechanism is used to characterize the mapping from the normal structure to its corresponding normal texture. As the correspondence between destroyed structure and texture cannot be characterized by the memory, the abnormal images would have a larger reconstruction error, facilitating anomaly detection. In this work, we utilize two kinds of complementary structures (i.e., the semantic structure with human-labeled category information and the low-level structure with abundant details), which are extracted by two structure extractors. The reconstructions from the two kinds of structures are fused together by a learned attention weight to get the final reconstructed image. We further feed the reconstructed image into the two aforementioned structure extractors to extract structures. On the one hand, constraining the consistency between the structures extracted from the original input and that from the reconstructed image would regularize the network training; on the other hand, the error between the structures extracted from the original input and that from the reconstructed image can also be used as a supplement measurement to identify the anomaly. Extensive experiments validate the effectiveness of our method for image anomaly detection on both industrial inspection images and medical images.
28 citations
TL;DR: Wang et al. as discussed by the authors used an image alignment method to generate sequences of AS-OCT images and then localized the anterior chamber angle region automatically by segmenting an important biomarker -the iris -as this is a primary structural cue in identifying angle-closure disease.
27 citations
TL;DR: Zhang et al. as mentioned in this paper proposed an end-to-end framework, called Attention-based Triplet Hashing (ATH) network, to learn low-dimensional hash codes that preserve the classification, ROI, and small-sample information.
22 citations
13 Apr 2021
TL;DR: Li et al. as mentioned in this paper proposed an end-to-end unsupervised restoration method of cataract images to enhance the clinical observation of Cataract fundus, which is the leading cause of preventable blindness in the world.
Abstract: Cataract presents the leading cause of preventable blindness in the world. The degraded image quality of cataract fundus increases the risk of misdiagnosis and the uncertainty in preoperative planning. Unfortunately, the absence of annotated data, which should consist of cataract images and the corresponding clear ones from the same patients after surgery, limits the development of restoration algorithms for cataract images. In this paper, we propose an end-to-end unsupervised restoration method of cataract images to enhance the clinical observation of cataract fundus. The proposed method begins with constructing an annotated source domain through simulating cataract-like images. Then a restoration model for cataract images is designed based on pix2pix framework and trained via unsupervised domain adaptation to generalize the restoration mapping from simulated data to real one. In the experiment, the proposed method is validated in an ablation study and a comparison with previous methods. A favorable performance is presented by the proposed method against the previous methods. The code of of this paper will be released at https://github.com/liamheng/Restoration-of-Cataract-Images-via-Domain-Adaptation.
16 citations
13 Apr 2021
TL;DR: Wang et al. as mentioned in this paper proposed a 3D vessel reconstruction framework based on the estimation of vessel depth maps from OCTA images, which combines MSE and SSIM loss as the training loss function.
Abstract: Optical Coherence Tomography Angiography (OCTA) has been increasingly used in the management of eye and systemic diseases in recent years. Manual or automatic analysis of blood vessel in 2D OCTA images (en face angiograms) is commonly used in clinical practice, however it may lose rich 3D spatial distribution information of blood vessels or capillaries that are useful for clinical decision-making. In this paper, we introduce a novel 3D vessel reconstruction framework based on the estimation of vessel depth maps from OCTA images. First, we design a network with structural constraints to predict the depth of blood vessels in OCTA images. In order to promote the accuracy of the predicted depth map at both the overall structure- and pixel- level, we combine MSE and SSIM loss as the training loss function. Finally, the 3D vessel reconstruction is achieved by utilizing the estimated depth map and 2D vessel segmentation results. Experimental results demonstrate that our method is effective in the depth prediction and 3D vessel reconstruction for OCTA images.
7 citations
TL;DR: Zhang et al. as discussed by the authors proposed a level set based deep learning method for optic disc and cup segmentation, which treated the output of the neural network as a level-set and added several constraints to make the predicted level set satisfy some characteristics, such as the length constraint and region constraint.
7 citations
TL;DR: Zhang et al. as discussed by the authors proposed a novel deep framework called Y-Net, which can learn highly discriminative convolutional features by unifying the pixel-wise segmentation loss and classification loss.
Abstract: When encountering a dubious diagnostic case, medical instance retrieval can help radiologists make evidence-based diagnoses by finding images containing instances similar to a query case from a large image database. The similarity between the query case and retrieved similar cases is determined by visual features extracted from pathologically abnormal regions. However, the manifestation of these regions often lacks specificity, i.e., different diseases can have the same manifestation, and different manifestations may occur at different stages of the same disease. To combat the manifestation ambiguity in medical instance retrieval, we propose a novel deep framework called Y-Net, encoding images into compact hash-codes generated from convolutional features by feature aggregation. Y-Net can learn highly discriminative convolutional features by unifying the pixel-wise segmentation loss and classification loss. The segmentation loss allows exploring subtle spatial differences for good spatial-discriminability while the classification loss utilizes class-aware semantic information for good semantic-separability. As a result, Y-Net can enhance the visual features in pathologically abnormal regions and suppress the disturbing of the background during model training, which could effectively embed discriminative features into the hash-codes in the retrieval stage. Extensive experiments on two medical image datasets demonstrate that Y-Net can alleviate the ambiguity of pathologically abnormal regions and its retrieval performance outperforms the state-of-the-art method by an average of 9.27% on the returned list of 10.
TL;DR: In this paper, a feature extractor equipped with a multi-scale attention module was proposed to learn global attention maps from global images, and a well-trained pixel-wise segmentation model was used to generate binarization masks.
Abstract: Chest X-rays are the most commonly available and affordable radiological examination for screening thoracic diseases. According to the domain knowledge of screening chest X-rays, the pathological information usually lay on the lung and heart regions. However, it is costly to acquire region-level annotation in practice, and model training mainly relies on image-level class labels in a weakly supervised manner, which is highly challenging for computer-aided chest X-ray screening. To address this issue, some methods have been proposed recently to identify local regions containing pathological information, which is vital for thoracic disease classification. Inspired by this, we propose a novel deep learning framework to explore discriminative information from lung and heart regions. We design a feature extractor equipped with a multi-scale attention module to learn global attention maps from global images. To exploit disease-specific cues effectively, we locate lung and heart regions containing pathological information by a well-trained pixel-wise segmentation model to generate binarization masks. By introducing element-wise logical AND operator on the learned global attention maps and the binarization masks, we obtain local attention maps in which pixels are are 1 for lung and heart region and 0 for other regions. By zeroing features of non-lung and heart regions in attention maps, we can effectively exploit their disease-specific cues in lung and heart regions. Compared to existing methods fusing global and local features, we adopt feature weighting to avoid weakening visual cues unique to lung and heart regions. Our method with pixel-wise segmentation can help overcome the deviation of locating local regions. Evaluated by the benchmark split on the publicly available chest X-ray14 dataset, the comprehensive experiments show that our method achieves superior performance compared to the state-of-the-art methods. We propose a novel deep framework for the multi-label classification of thoracic diseases in chest X-ray images. The proposed network aims to effectively exploit pathological regions containing the main cues for chest X-ray screening. Our proposed network has been used in clinic screening to assist the radiologists. Chest X-ray accounts for a significant proportion of radiological examinations. It is valuable to explore more methods for improving performance.
TL;DR: In this paper, the authors examined the macular microvascular changes of the macula in neuromyelitis optica spectrum disorder (NMOSD) patients and its association with their disability and other clinical variables.
Abstract: Purpose: We examined the macular microvascular changes of the macula in neuromyelitis optica spectrum disorder (NMOSD) patients and its association with their disability and other clinical variables.
Methods: Thirty-four NMOSD (13 patients without optic neuritis, NMOSD-NON, and 21 patients with a history of optic neuritis, NMOSD-ON) and 44 healthy controls (HCs) were included in the study. Optical coherence tomographic angiography (OCTA) was used to image the superficial (SCP), deep (DCP), and whole capillary plexus (WCP) in a 2.5-mm-diameter concentric circle [excluding the foveal avascular zone (FAZ)]. An algorithm (Dbox) was used to quantify the complexity of the three capillary layers by fractal analysis. We also evaluated the expanded disability scale status (EDSS).
Results: Dbox values were significantly reduced in SCP (p < 0.001), DCP (p < 0.001), and WCP (p = 0.003) of NMOSD when compared with HCs. Dbox values were significantly reduced in NMOSD eyes with optic neuritis when compared with healthy controls (p < 0.001) and eyes without optic neuritis (p = 0.004) in the SCP. In the DCP, eyes with optic neuritis showed significantly reduced Dbox values when compared with eyes without optic neuritis (p = 0.016) and healthy controls (p < 0.001); eyes without optic neuritis showed significantly reduced Dbox values (p = 0.007) in the DCP when compared with healthy controls. A significant negative correlation (Rho = −0.475, p = 0.005) was shown between the superficial macula Dbox values and the EDSS in NMOSD patients. Additionally, a negative correlation (Rho = −0.715, p = 0.006) was seen in the superficial Dbox values in [e]eyes without optic neuritis and EDSS.
Conclusions: Macular microvascular damage in the superficial plexus is associated with disability in NMOSD. Macular microvascular alterations arise independently of the occurrence of ON in NMOSD.
27 Sep 2021
TL;DR: Wang et al. as mentioned in this paper proposed a 2D-to-3D vessel reconstruction framework based on the 2D en face OCTA images, which takes advantage of the detailed 2D OCTA depth map for prediction and thus does not rely on any 3D volumetric data.
Abstract: Optical Coherence Tomography Angiography (OCTA) has been widely used by ophthalmologists for decision-making due to its superiority in providing caplillary details. Many of the OCTA imaging devices used in clinic provide high-quality 2D en face representations, while their 3D data quality are largely limited by low signal-to-noise ratio and strong projection artifacts, which restrict the performance of depth-resolved 3D analysis. In this paper, we propose a novel 2D-to-3D vessel reconstruction framework based on the 2D en face OCTA images. This framework takes advantage of the detailed 2D OCTA depth map for prediction and thus does not rely on any 3D volumetric data. Based on the data with available vessel depth labels, we first introduce a network with structure constraint blocks to estimate the depth map of blood vessels in other cross-domain en face OCTA data with unavailable labels. Afterwards, a depth adversarial adaptation module is proposed for better unsupervised cross-domain training, since images captured using different devices may suffer from varying image contrast and noise levels. Finally, vessels are reconstructed in 3D space by utilizing the estimated depth map and 2D vascular information. Experimental results demonstrate the effectiveness of our method and its potential to guide subsequent vascular analysis in 3D domain.
TL;DR: Usception as discussed by the authors is a 3D U-Net-like framework for small cerebrovascular segmentation, which includes three blocks: Reduction block, Gap block, and Deep block, aiming to improve feature extraction ability by grouping different convolution sizes.
Abstract: Purpose Cerebrovascular segmentation in magnetic resonance imaging (MRI) plays an important role in the diagnosis and treatment of cerebrovascular diseases. Many segmentation frameworks based on convolutional neural networks (CNNs) or U-Net-like structures have been proposed for cerebrovascular segmentation. Unfortunately, the segmentation results are still unsatisfactory, particularly in the small/thin cerebrovascular due to the following reasons: (1) the lack of attention to multiscale features in encoder caused by the convolutions with single kernel size; (2) insufficient extraction of shallow and deep-seated features caused by the depth limitation of transmission path between encoder and decoder; (3) insufficient utilization of the extracted features in decoder caused by less attention to multiscale features. Methods Inspired by U-Net++, we propose a novel 3D U-Net-like framework termed Usception for small cerebrovascular. It includes three blocks: Reduction block, Gap block, and Deep block, aiming to: (1) improve feature extraction ability by grouping different convolution sizes; (2) increase the number of multiscale features in different layers by grouping paths of different depths between encoder and decoder; (3) maximize the ability of decoder in recovering multiscale features from Reduction and Gap block by using convolutions with different kernel sizes. Results The proposed framework is evaluated on three public and in-house clinical magnetic resonance angiography (MRA) data sets. The experimental results show that our framework reaches an average dice score of 69.29%, 87.40%, 77.77% on three data sets, which outperform existing state-of-the-art methods. We also validate the effectiveness of each block through ablation experiments. Conclusions By means of the combination of Inception-ResNet and dimension-expanded U-Net++, the proposed framework has demonstrated its capability to maximize multiscale feature extraction, thus achieving competitive segmentation results for small cerebrovascular.
TL;DR: The BREATHE study as mentioned in this paper is the first study to implement a comprehensive risk-based mammography screening program in Asia, which integrates both genetic and non-genetic breast cancer risk prediction tools to personalise screening recommendations.
Abstract: BackgroundRoutine mammography screening is currently the standard tool for finding cancers at an early stage, when treatment is most successful. Current breast screening programmes are one-size-fits-all which all women above a certain age threshold are encouraged to participate. However, breast cancer risk varies by individual. The BREAst screening Tailored for HEr (BREATHE) study aims to assess acceptability of a comprehensive risk-based personalised breast screening in Singapore. Methods/DesignAdvancing beyond the current age-based screening paradigm, BREATHE integrates both genetic and non-genetic breast cancer risk prediction tools to personalise screening recommendations. BREATHE is a cohort study targeting to recruit [~]3,500 women. The first recruitment visit will include questionnaires and a buccal cheek swab. After receiving a tailored breast cancer risk report, participants will attend an in-person risk review, followed by a final session assessing the acceptability of our risk stratification programme. Risk prediction is based on: a) Gail model (non-genetic), b) mammographic density and recall, c) BOADICEA predictions (breast cancer predisposition genes), and d) breast cancer polygenic risk score. DiscussionFor national implementation of personalised risk-based breast screening, exploration of the acceptability within the target populace is critical, in addition to validated predication tools. To our knowledge, this is the first study to implement a comprehensive risk-based mammography screening programme in Asia. The BREATHE study will provide essential data for policy implementation which will transform the health system to deliver a better health and healthcare outcomes. Trial registrationNot applicable.
Posted Content•
TL;DR: In this paper, a spatial context-aware deep neural network is proposed to predict labels taking into account both semantic and spatial information for multi-label image classification, and the proposed framework is evaluated on Microsoft COCO and PASCAL VOC, two widely used benchmark datasets.
Abstract: Multi-label image classification is a fundamental but challenging task in computer vision. Over the past few decades, solutions exploring relationships between semantic labels have made great progress. However, the underlying spatial-contextual information of labels is under-exploited. To tackle this problem, a spatial-context-aware deep neural network is proposed to predict labels taking into account both semantic and spatial information. This proposed framework is evaluated on Microsoft COCO and PASCAL VOC, two widely used benchmark datasets for image multi-labelling. The results show that the proposed approach is superior to the state-of-the-art solutions on dealing with the multi-label image classification problem.
TL;DR: In this article, the authors investigated the thickness changes of outer retinal layers in subjects with white matter hyperintensities (WMH) and Parkinson's disease (PD) and found that WMH patients had significantly thinner Henle fiber layers, outer nuclear layers (HFL+ONL) and photoreceptor outer segments (OS) than HC subjects.
Abstract: Purpose: To investigate the thickness changes of outer retinal layers in subjects with white matter hyperintensities (WMH) and Parkinson's Disease (PD). Methods: 56 eyes from 31 patients with WMH, 11 eyes from 6 PD patients, and 58 eyes from 32 healthy controls (HC) were enrolled in this study. A macular-centered scan was conducted on each participant using a spectral-domain optical coherence tomography (SD-OCT) device. After speckle noise reduction, a state-of-the-art deep learning method (i.e., a context encoder network) was employed to segment the outer retinal layers from OCT B-scans. Thickness quantification of the outer retinal layers was conducted on the basis of the segmentation results. Results: WMH patients had significantly thinner Henle fiber layers, outer nuclear layers (HFL+ONL) and photoreceptor outer segments (OS) than HC (p = 0.031, and p = 0.005), while PD patients showed a significant increase of mean thickness in the interdigitation zone and the retinal pigment epithelium/Bruch complex (IZ+RPE) (19.619 ± 4.626) compared to HC (17.434 ± 1.664). There were no significant differences in the thickness of the outer plexiform layer (OPL), the myoid and ellipsoid zone (MEZ), and the IZ+RPE layer between WMH and HC subjects. Similarly, there were also no obvious differences in the thickness of the OPL, HFL+ONL, MEZ and the OS layer between PD and HC subjects. Conclusion: Thickness changes in HFL+ONL, OS, and IZ+RPE layers may correlate with brain-related diseases such as WMH and PD. Further longitudinal study is needed to confirm HFL+ONL/OS/IZ+RPE layer thickness as potential biomarkers for detecting certain brain-related diseases.
Posted Content•
TL;DR: Wang et al. as mentioned in this paper proposed a multi-branch hybrid Transformer Network (MBT-Net) based on the transformer and body-edgebranch to segment corneal endothelial cells.
Abstract: Corneal endothelial cell segmentation plays a vital role inquantifying clinical indicators such as cell density, coefficient of variation,and hexagonality. However, the corneal endothelium's uneven reflectionand the subject's tremor and movement cause blurred cell edges in theimage, which is difficult to segment, and need more details and contextinformation to release this problem. Due to the limited receptive field oflocal convolution and continuous downsampling, the existing deep learn-ing segmentation methods cannot make full use of global context andmiss many details. This paper proposes a Multi-Branch hybrid Trans-former Network (MBT-Net) based on the transformer and body-edgebranch. Firstly, We use the convolutional block to focus on local tex-ture feature extraction and establish long-range dependencies over space,channel, and layer by the transformer and residual connection. Besides,We use the body-edge branch to promote local consistency and to provideedge position information. On the self-collected dataset TM-EM3000 andpublic Alisarine dataset, compared with other State-Of-The-Art (SOTA)methods, the proposed method achieves an improvement.
TL;DR: Wang et al. as mentioned in this paper proposed an adaptive 3D shearlet image filter with noise-redistribution (adaptive-SIN) scheme for OCT images, which effectively transforms the Poisson noise to Gaussian noise.
Abstract: Optical coherence tomography (OCT) images is widely used in ophthalmic examination, but their qualities are often affected by noises. Shearlet transform has shown its effectiveness in removing image noises because of its edge-preserving property and directional sensitivity. In the paper, we propose an adaptive denoising algorithm for OCT images. The OCT noise is closer to the Poisson distribution than the Gaussian distribution, and shearlet transform assumes additive white Gaussian noise. We hence propose a square-root transform to redistribute the OCT noise. Different manufacturers and differences between imaging objects may influence the observed noise characteristics, which make predefined thresholding scheme ineffective. We propose an adaptive 3D shearlet image filter with noise-redistribution (adaptive-SIN) scheme for OCT images. The proposed adaptive-SIN is evaluated on three benchmark datasets using quantitative evaluation metrics and subjective visual inspection. Compared with other algorithms, the proposed algorithm better removes noise in OCT images and better preserves image details, significantly outperforming in terms of both quantitative evaluation and visual inspection. The proposed algorithm effectively transforms the Poisson noise to Gaussian noise so that the subsequent shearlet transform could optimally remove the noise. The proposed adaptive thresholding scheme optimally adapts to various noise conditions and hence better remove the noise. The comparison experimental results on three benchmark datasets against 8 compared algorithms demonstrate the effectiveness of the proposed approach in removing OCT noise.
DOI•
13 Aug 2021
TL;DR: In this article, a fuzzy integral based ensemble framework of multiple deep learning models for optic disc segmentation is proposed, where each component segmentation model is trained with respect to an annotator and a powerful nonlinear aggregation function is employed in form of a neural network to integrate the segmentation results of multiple annotators.
Abstract: Modern deep neural networks are able to beat human annotators in several medical image processing tasks. In practical manual annotation for medical image segmentation tasks, the labels of annotators often show inter-observer variability (IOV) which is mainly caused by annotators’ different understandings of expertise. In order to build a trustworthy segmentation system, robust models should consider how to capture uncertainty in samples and labels. Different from the conventional way of handling IOV with label fusion such as majority voting, a fuzzy integral based ensemble framework of multiple deep learning models for optic disc segmentation is proposed. Each component segmentation model is trained with respect to an annotator. Then, a powerful nonlinear aggregation function, the Choquet integral, is employed in form of a neural network to integrate the segmentation results of multiple annotators. The proposed method is validated on the public RIM-ONE dataset consisting of 169 fundus images and each image is annotated by 5 experts. Compared with conventional segmentation ensemble methods, the proposed methods achieves a higher Dice score (98.69%).
13 Apr 2021
TL;DR: In this article, a memory-assisted dual-end adaptation network is proposed to address the universality problem in OCT choroid segmentation by memorizing the encoded style features of every involved domain.
Abstract: Accurate measurement of choroid layer in optical coherence tomography (OCT) is crucial in the diagnosis of many ocular diseases, such as pathological myopia and glaucoma. Deep learning has shown its superiority in automatic choroid segmentation. However, because of the domain discrepancies among datasets obtained by the OCT devices of different manufacturers, the generalization capability of trained models is limited. We propose a memory-assisted dual-end adaptation network to address the universality problem. Different from the existing works that can only perform one-to-one domain adaptation, our method is capable of performing one-to-many adaptation. In the proposed method, we introduce a memory module to memorize the encoded style features of every involved domain. Both input and output space adaptation are employed to regularize the choroid segmentation. We evaluate the proposed method over different datasets acquired by four major OCT manufacturers (TOPCON, NIDEK, ZEISS, HEIDELBERG). Experiments show that our proposed method outperforms existing methods with significant margins of improvement in terms of all metrics.
27 Sep 2021
TL;DR: In this paper, a neural network algorithm was developed to predict the spherical equivalent (SE) using real-world clinical non-cycloplegic refraction records and domain knowledge.
Abstract: Traditional cycloplegic refractive power detection with specific lotions dropping may cause side-effects, e.g., the pupillary retraction disorder, on juvenile eyes. In this paper, we develop a novel neural network algorithm to predict the refractive power, which is assessed by the Spherical Equivalent (SE), using real-world clinical non-cycloplegic refraction records. Participants underwent a comprehensive ophthalmic examination to obtain several related parameters, including sphere degree, cylinder degree, axial length, flat keratometry, and steep keratometry. Based on these quantitative biomedical parameters, a novel neural network model is trained to predict the SE. On the whole age test dataset, the domain knowledge embedding network (DKE-Net) prediction accuracies of SE achieve 59.82% (between \(\pm 0.5D\)), 86.85% (between \(\pm 1D\)), 95.54% (between \(\pm 1.5D\)), and 98.57% (between \(\pm 2D\)), which demonstrate superior performance over conventional machine learning algorithms on real-world clinical electronic refraction records. Also, the SE prediction accuracies on the excluded examples that are disqualified for model training, are 2.16% (between \(\pm 0.5D\)), 3.76% (between \(\pm 1D\)), 6.15% (between \(\pm 1.5D\)), and 8.78% (between \(\pm 2D\)). This is the leading application to predict refraction power using a neural network and domain knowledge, to the best of our knowledge, with a satisfactory accuracy level. Moreover, the model can also assist in diagnosing some specific kinds of ocular disorders.
27 Sep 2021
TL;DR: Zhang et al. as mentioned in this paper proposed a new unsupervised guided adversarial adaptation (GAA) network to segment both retinal layers and the choroid in OCT images.
Abstract: Morphological changes, e.g. thickness of retinal or choroidal layers in Optical coherence tomography (OCT), is of great importance in clinic applications as they reveal some specific eye diseases and other systemic conditions. However, there are many challenges in the accurate segmentation of retinal and choroidal layers, such as low contrast between different tissue layers and variations between images acquired from multiple devices. There is a strong demand on accurate and robust segmentation models with high generalization ability to deal with images from different devices. This paper proposes a new unsupervised guided adversarial adaptation (GAA) network to segment both retinal layers and the choroid in OCT images. To our best knowledge, this is the first work to extract retinal and choroidal layers in a unified manner. It first introduces a dual encoder structure to ensure that the encoding path of the source domain image is independent of that of the target domain image. By integrating the dual encoder into an adversarial framework, the holistic GAA network significantly alleviates the performance degradation of the source domain image segmentation caused by parameter entanglement with the encoder of the target domain and also improves the segmentation performance of the target domain images. Experimental results show that the proposed network outperforms other state-of-the-art methods in retinal and choroidal layer segmentation.
DOI•
13 Aug 2021TL;DR: Li et al. as mentioned in this paper presented a deep learning network with a multi-scale self-attention module to aggregate the global context to learned features for diabetic retinopathy (DR) image retrieval.
Abstract: Diabetic retinopathy (DR), a complication due to diabetes, is a common cause of progressive damage to the retina. The mass screening of populations for DR is time-consuming. Therefore, computerized diagnosis is of great significance in the clinical practice, which providing evidence to assist clinicians in decision making. Specifically, hemorrhages, microaneurysms, hard exudates, soft exudates, and other lesions are verified to be closely associated with DR. These lesions, however, are scattered in different positions and sizes in fundus images, the internal relation of which are hard to be reserved in the ultimate features due to a large number of convolution layers that reduce the detail characteristics. In this paper, we present a deep-learning network with a multi-scale self-attention module to aggregate the global context to learned features for DR image retrieval. The multi-scale fusion enhances, in terms of scale, the efficacious latent relation of different positions in features explored by the self-attention. For the experiment, the proposed network is validated on the Kaggle DR dataset, and the result shows that it achieves state-of-the-art performance.
24 May 2021
TL;DR: This chapter proposes a fuzzy pattern tree-based approach for the automated grading of corneal nerves’ tortuosity based on IVCM images, which starts with the deep learning-based image segmentation of corNeal nerves and then extracts several morphological Tortuosity measurements as features for further processing.
Abstract: The tortuosity of corneal nerve fibers is correlated with a number of diseases such as diabetic neuropathy. The assessment of corneal nerve tortuosity level in in vivo confocal microscopy (IVCM) images can inform the detection of early diseases and further complications. With the aim to assess the corneal nerve tortuosity accurately as well as to extract knowledge meaningful to ophthalmologists, this chapter proposes a fuzzy pattern tree-based approach for the automated grading of corneal nerves’ tortuosity based on IVCM images. The proposed method starts with the deep learning-based image segmentation of corneal nerves and then extracts several morphological tortuosity measurements as features for further processing. Finally, the fuzzy pattern trees are constructed based on the extracted features for the tortuosity grading. Experimental results on a public corneal nerve data set demonstrate the effectiveness of fuzzy pattern tree in IVCM image tortuosity assessment.
Posted Content•
TL;DR: Zhang et al. as mentioned in this paper designed a feature extractor equipped with a multi-scale attention module to learn global attention maps from global images and locate lung and heart regions containing pathological information by a well-trained pixelwise segmentation model to generate binarization masks.
Abstract: Chest X-rays are the most commonly available and affordable radiological examination for screening thoracic diseases. According to the domain knowledge of screening chest X-rays, the pathological information usually lay on the lung and heart regions. However, it is costly to acquire region-level annotation in practice, and model training mainly relies on image-level class labels in a weakly supervised manner, which is highly challenging for computer-aided chest X-ray screening. To address this issue, some methods have been proposed recently to identify local regions containing pathological information, which is vital for thoracic disease classification. Inspired by this, we propose a novel deep learning framework to explore discriminative information from lung and heart regions. We design a feature extractor equipped with a multi-scale attention module to learn global attention maps from global images. To exploit disease-specific cues effectively, we locate lung and heart regions containing pathological information by a well-trained pixel-wise segmentation model to generate binarization masks. By introducing element-wise logical AND operator on the learned global attention maps and the binarization masks, we obtain local attention maps in which pixels are $1$ for lung and heart region and $0$ for other regions. By zeroing features of non-lung and heart regions in attention maps, we can effectively exploit their disease-specific cues in lung and heart regions. Compared to existing methods fusing global and local features, we adopt feature weighting to avoid weakening visual cues unique to lung and heart regions. Evaluated by the benchmark split on the publicly available chest X-ray14 dataset, the comprehensive experiments show that our method achieves superior performance compared to the state-of-the-art methods.
TL;DR: Wang et al. as mentioned in this paper investigated the ocular biometric parameters, peripheral refraction, and accommodative lag of fellow eyes in anisometropic Chinese children, and found that anterior and posterior biometrics differ in many aspects between fellow eyes of Chinese children.
Abstract: SIGNIFICANCE This research found that anterior and posterior biometrics differ in many aspects between fellow eyes of anisometropic children. This might shed light on the mechanisms underlying the onset and progression of anisometropia and myopia. PURPOSE This study aimed to investigate the ocular biometric parameters, peripheral refraction, and accommodative lag of fellow eyes in anisometropic children. METHODS Anisometropic children were recruited. Axial length (AL), vitreous chamber depth (VCD), central corneal thickness, anterior chamber depth (ACD), lens thickness (LT), simulated K readings, central and peripheral refractive errors, and accommodative lag were measured in both eyes. The subfoveal choroidal thickness, average choroidal thickness, and choroid vessel density of the 6 × 6-mm macular area were measured by optical coherence tomography. RESULTS Thirty-two children aged 11.1 ± 1.7 years were enrolled. The average degree of anisometropia was 2.49 ± 0.88 D. The AL, VCD, ACD, and simulated K reading values were significantly larger in the more myopic eyes, whereas the LT value was significantly smaller. Subfoveal choroidal thickness (P = .001) and average choroidal thickness (P = .02) were smaller in the more myopic eyes than in the contralateral eyes, whereas choroid vessel density (P = .03) was larger. The amount of anisometropia had a significant positive correlation with the difference in AL (r = 0.869, P < .001), VCD (r = 0.853, P < .001), and ACD (r = 0.591, P < .001) and a negative correlation with the difference in LT (r = -0.457, P = .009). CONCLUSIONS Ocular biometrics differ in many aspects between the fellow eyes of anisometropic Chinese children, and the difference is correlated with the degree of anisometropia.
Posted Content•
TL;DR: Zhang et al. as discussed by the authors proposed a novel deep framework called Y-Net, which can learn highly discriminative convolutional features by unifying the pixel-wise segmentation loss and classification loss.
Abstract: When encountering a dubious diagnostic case, medical instance retrieval can help radiologists make evidence-based diagnoses by finding images containing instances similar to a query case from a large image database. The similarity between the query case and retrieved similar cases is determined by visual features extracted from pathologically abnormal regions. However, the manifestation of these regions often lacks specificity, i.e., different diseases can have the same manifestation, and different manifestations may occur at different stages of the same disease. To combat the manifestation ambiguity in medical instance retrieval, we propose a novel deep framework called Y-Net, encoding images into compact hash-codes generated from convolutional features by feature aggregation. Y-Net can learn highly discriminative convolutional features by unifying the pixel-wise segmentation loss and classification loss. The segmentation loss allows exploring subtle spatial differences for good spatial-discriminability while the classification loss utilizes class-aware semantic information for good semantic-separability. As a result, Y-Net can enhance the visual features in pathologically abnormal regions and suppress the disturbing of the background during model training, which could effectively embed discriminative features into the hash-codes in the retrieval stage. Extensive experiments on two medical image datasets demonstrate that Y-Net can alleviate the ambiguity of pathologically abnormal regions and its retrieval performance outperforms the state-of-the-art method by an average of 9.27\% on the returned list of 10.
Posted Content•
TL;DR: Wang et al. as mentioned in this paper proposed a 3D vessel reconstruction framework based on the estimation of vessel depth maps from OCTA images, which combines MSE and SSIM loss as the training loss function.
Abstract: Optical Coherence Tomography Angiography (OCTA) has been increasingly used in the management of eye and systemic diseases in recent years. Manual or automatic analysis of blood vessel in 2D OCTA images (en face angiograms) is commonly used in clinical practice, however it may lose rich 3D spatial distribution information of blood vessels or capillaries that are useful for clinical decision-making. In this paper, we introduce a novel 3D vessel reconstruction framework based on the estimation of vessel depth maps from OCTA images. First, we design a network with structural constraints to predict the depth of blood vessels in OCTA images. In order to promote the accuracy of the predicted depth map at both the overall structure- and pixel- level, we combine MSE and SSIM loss as the training loss function. Finally, the 3D vessel reconstruction is achieved by utilizing the estimated depth map and 2D vessel segmentation results. Experimental results demonstrate that our method is effective in the depth prediction and 3D vessel reconstruction for OCTA images.% results may be used to guide subsequent vascular analysis