Book ChapterDOI
COMe-SEE: Cross-modality Semantic Embedding Ensemble for Generalized Zero-Shot Diagnosis of Chest Radiographs
Angshuman Paul,Thomas C. Shen,Niranjan Balachandar,Yuxing Tang,Yifan Peng,Zhiyong Lu,Ronald M. Summers +6 more
- pp 103-111
Reads0
Chats0
TLDR
This work introduces a first-of-its-kind generalized zero-shot learning (GZSL) framework that utilizes information from two different imaging modalities (CT and x-ray) for the diagnosis of chest radiographs and makes use of CT radiology reports to create a semantic space consisting of signatures corresponding to different chest diseases and conditions.Abstract:
Zero-shot learning, in spite of its recent popularity, remains an unexplored area for medical image analysis. We introduce a first-of-its-kind generalized zero-shot learning (GZSL) framework that utilizes information from two different imaging modalities (CT and x-ray) for the diagnosis of chest radiographs. Our model makes use of CT radiology reports to create a semantic space consisting of signatures corresponding to different chest diseases and conditions. We introduce a CrOss-Modality Semantic Embedding Ensemble (COMe-SEE) for zero-shot diagnosis of chest x-rays by relating an input x-ray to a signature in the semantic space. The ensemble, designed using a novel semantic saliency preserving autoencoder, utilizes the visual and the semantic saliency to facilitate GZSL. The use of an ensemble not only helps in dealing with noise but also makes our model useful across different datasets. Experiments on two publicly available datasets show that the proposed model can be trained using one dataset and still be applied to data from another source for zero-shot diagnosis of chest x-rays.read more
Citations
More filters
Journal ArticleDOI
Generalized Zero-Shot Chest X-Ray Diagnosis Through Trait-Guided Multi-View Semantic Embedding With Self-Training
Angshuman Paul,Thomas C. Shen,Sungwon Lee,Niranjan Balachandar,Yifan Peng,Zhiyong Lu,Ronald M. Summers +6 more
TL;DR: In this article, a multi-view semantic embedding strategy for generalized zero-shot diagnosis of chest radiographs is proposed, which leverages the potential of multiview semantic embeddings.
References
More filters
Journal ArticleDOI
Random Forests
TL;DR: Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the forest, and are also applicable to regression.
Proceedings Article
DeViSE: A Deep Visual-Semantic Embedding Model
Andrea Frome,Greg S. Corrado,Jon Shlens,Samy Bengio,Jeffrey Dean,Marc'Aurelio Ranzato,Tomas Mikolov +6 more
TL;DR: This paper presents a new deep visual-semantic embedding model trained to identify visual objects using both labeled image data as well as semantic information gleaned from unannotated text and shows that the semantic information can be exploited to make predictions about tens of thousands of image labels not observed during training.
Proceedings ArticleDOI
ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
TL;DR: The ChestX-ray dataset as discussed by the authors contains 108,948 frontal-view X-ray images of 32,717 unique patients with the text-mined eight disease image labels from the associated radiological reports using natural language processing.
Book ChapterDOI
An embarrassingly simple approach to zero-shot learning
TL;DR: This paper describes a zero-shot learning approach that can be implemented in just one line of code, yet it is able to outperform state of the art approaches on standard datasets.
Proceedings ArticleDOI
ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
TL;DR: A new chest X-rays database, namely ChestX-ray8, is presented, which comprises 108,948 frontal-view X-ray images of 32,717 unique patients with the text-mined eight disease image labels from the associated radiological reports using natural language processing, which is validated using the proposed dataset.