When we are faced with challenging image classification tasks, we often explain our reasoning by dissecting the image, and pointing out prototypical aspects of one class or another. The mounting evidence for each of the classes helps us make our final decision. In this work, we introduce a deep network architecture -- prototypical part network (ProtoPNet), that reasons in a similar way: the network dissects the image by finding prototypical parts, and combines evidence from the prototypes to make a final classification. The model thus reasons in a way that is qualitatively similar to the way ornithologists, physicians, and others would explain to people on how to solve challenging image classification tasks. The network uses only image-level labels for training without any annotations for parts of images. We demonstrate our method on the CUB-200-2011 dataset and the Stanford Cars dataset. Our experiments show that ProtoPNet can achieve comparable accuracy with its analogous non-interpretable counterpart, and when several ProtoPNets are combined into a larger network, it can achieve an accuracy that is on par with some of the best-performing deep models. Moreover, ProtoPNet provides a level of interpretability that is absent in other interpretable deep models.

This Looks Like That: Deep Learning for Interpretable Image Recognition

In this study, we evaluated the ability of radiomic textural analysis of intratumoral and peritumoral regions on pretreatment breast cancer dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) to predict pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) A total of 117 patients who had received NAC were retrospectively analyzed Within the intratumoral and peritumoral regions of T1-weighted contrast-enhanced MRI scans, a total of 99 radiomic textural features were computed at multiple phases Feature selection was used to identify a set of top pCR-associated features from within a training set (n = 78), which were then used to train multiple machine learning classifiers to predict the likelihood of pCR for a given patient Classifiers were then independently tested on 39 patients Experiments were repeated separately among hormone receptor-positive and human epidermal growth factor receptor 2-negative (HR+, HER2−) and triple-negative or HER2+ (TN/HER2+) tumors via threefold cross-validation to determine whether receptor status-specific analysis could improve classification performance Among all patients, a combined intratumoral and peritumoral radiomic feature set yielded a maximum AUC of 078 ± 0030 within the training set and 074 within the independent testing set using a diagonal linear discriminant analysis (DLDA) classifier Receptor status-specific feature discovery and classification enabled improved prediction of pCR, yielding maximum AUCs of 083 ± 0025 within the HR+, HER2− group using DLDA and 093 ± 0018 within the TN/HER2+ group using a naive Bayes classifier In HR+, HER2− breast cancers, non-pCR was characterized by elevated peritumoral heterogeneity during initial contrast enhancement However, TN/HER2+ tumors were best characterized by a speckled enhancement pattern within the peritumoral region of nonresponders Radiomic features were found to strongly predict pCR independent of choice of classifier, suggesting their robustness as response predictors Through a combined intratumoral and peritumoral radiomics approach, we could successfully predict pCR to NAC from pretreatment breast DCE-MRI, both with and without a priori knowledge of receptor status Further, our findings suggest that the radiomic features most predictive of response vary across different receptor subtypes

/pdf/intratumoral-and-peritumoral-radiomics-for-the-pretreatment-di3vcdprm0.pdf

Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI

With the growing use of graph convolutional neural networks (GCNNs) comes the need for explainability. In this paper, we introduce explainability methods for GCNNs. We develop the graph analogues of three prominent explainability methods for convolutional neural networks: contrastive gradient-based (CG) saliency maps, Class Activation Mapping (CAM), and Excitation Back-Propagation (EB) and their variants, gradient-weighted CAM (Grad-CAM) and contrastive EB (c-EB). We show a proof-of-concept of these methods on classification problems in two application domains: visual scene graphs and molecular graphs. To compare the methods, we identify three desirable properties of explanations: (1) their importance to classification, as measured by the impact of occlusions, (2) their contrastivity with respect to different classes, and (3) their sparseness on a graph. We call the corresponding quantitative metrics fidelity, contrastivity, and sparsity and evaluate them for each method. Lastly, we analyze the salient subgraphs obtained from explanations and report frequently occurring patterns.

/pdf/explainability-methods-for-graph-convolutional-neural-1iehersnap.pdf

Explainability Methods for Graph Convolutional Neural Networks

In a short period of time, many areas of science have made a sharp transition towards data-dependent methods. In some cases, this process has been enabled by simultaneous advances in data acquisition and the development of networked system technologies. This new situation is particularly clear in the life sciences, where data overabundance has sparked a flurry of new methodologies for data management and analysis. This can be seen as a perfect scenario for the use of machine learning and computational intelligence techniques to address problems in which more traditional data analysis approaches might struggle. But, this scenario also poses some serious challenges. One of them is model interpretability and explainability, especially for complex nonlinear models. In some areas such as medicine and health care, not addressing such challenge might seriously limit the chances of adoption, in real practice, of computer-based systems that rely on machine learning and computational intelligence methods for data analysis. In this paper, we reflect on recent investigations about the interpretability and explainability of machine learning methods and discuss their impact on medicine and health care. We pay specific attention to one of the ways in which interpretability and explainability in this context can be addressed, which is through data and model visualization. We argue that, beyond improving model interpretability as a goal in itself, we need to integrate the medical experts in the design of data analysis interpretation strategies. Otherwise, machine learning is unlikely to become a part of routine clinical and health care practice.

/pdf/the-importance-of-interpretability-and-visualization-in-121ag95dhp.pdf

The importance of interpretability and visualization in machine learning for applications in medicine and health care

Purpose: We evaluated the performance of the newly proposed radiomics of multiparametric MRI (RMM), developed and validated based on a multicenter dataset adopting a radiomic strategy, for pretreatment prediction of pathologic complete response (pCR) to neoadjuvant chemotherapy (NAC) in breast cancer. Experimental Design: A total of 586 potentially eligible patients were retrospectively enrolled from four hospitals (primary cohort and external validation cohort 1–3). Quantitative imaging features were extracted from T2-weighted imaging, diffusion-weighted imaging, and contrast-enhanced T1-weighted imaging before NAC for each patient. With features selected using a coarse to fine feature selection strategy, four radiomic signatures were constructed based on each of the three MRI sequences and their combination. RMM was developed based on the best radiomic signature incorporating with independent clinicopathologic risk factors. The performance of RMM was assessed with respect to its discrimination and clinical usefulness, and compared with that of clinical information–based prediction model. Results: Radiomic signature combining multiparametric MRI achieved an AUC of 0.79 (the highest among the four radiomic signatures). The signature further achieved good performances in hormone receptor–positive and HER2-negative group and triple-negative group. RMM yielded an AUC of 0.86, which was significantly higher than that of clinical model in two of the three external validation cohorts. Conclusions: The study suggested a possibility that RMM provided a potential tool to develop a model for predicting pCR to NAC in breast cancer.

https://clincancerres.aacrjournals.org/content/clincanres/25/12/3538.full.pdf

Radiomics of Multiparametric MRI for Pretreatment Prediction of Pathologic Complete Response to Neoadjuvant Chemotherapy in Breast Cancer: A Multicenter Study.

Ectopic pregnancy is a major clinical problem, occurring in 75,000 cases per year in the United States. With in vitro fertilization, embryo transfer, microsurgical techniques, and better early diagnosis, the rates of ectopic pregnancies have increased. Transvaginal sonography along with beta-human chorionic gonadotrophin monitoring are the standards for evaluation of suspected ectopic pregnancy. When ectopic pregnancy is present, transvaginal sonography most often demonstrates an adnexal finding directly related to the extra-uterine gestation itself, a hematoma caused by it, or both. Although tubal location is most common, it is important to recognize unusual locations of ectopic pregnancy such as isthmic, cervical, ovarian, and abdominal. Ultrasound is the mainstay of diagnosis, but magnetic resonance imaging can be helpful in unusual or complicated cases. Current therapeutic options for an ectopic pregnancy are expectant management, systemic methotrexate injections, sonographically guided minimal invasive treatment, and surgery. This review encompasses diagnosis, treatment, and management of ectopic pregnancy.

Ectopic pregnancy: a review.

This prospective internally funded investigation demonstrates that the addition of digital breast tomosynthesis to a two-dimensional mammography screening program in a U.S. academic medical center resulted in a significant increase in the cancer detection rate, as well as a decrease in the screening recall rate overall and for patients with heterogeneous or extremely dense breasts and for patients in their 5th or 7th decades.

https://pubs.rsna.org/doi/pdf/10.1148/radiol.2015142036

Increased Cancer Detection Rate and Variations in the Recall Rate Resulting from Implementation of 3D Digital Breast Tomosynthesis into a Population-based Screening Program.

Neoadjuvant chemotherapy (NAC) involves administration of chemotherapeutic agents to patients with newly diagnosed breast cancer prior to definitive surgical treatment. Assessment of disease response to chemotherapeutic agents in vivo prior to any surgical intervention is necessary as medical oncologists are commonly tailoring or changing therapy during NAC based on response. It can also maximize the pathologic complete response (pCR) rate, resulting in more women undergoing breast conservation rather than mastectomy. Although some studies show a pCR to NAC in only 13–26 % of women, recent studies have shown higher pCR rates, especially for HER2-positive disease treated with targeted anti-HER2 therapy. Thus, accurate imaging tools for quantifying disease response are critical in the evaluation and management of patients undergoing NAC. There is currently no standard imaging method for monitoring response to therapy. Response to therapy tends to vary by tumor subtype and can be accurately assessed on imaging. We review the role of imaging before and after neoadjuvant therapy and discuss the advantages and limitations of currently available modalities, including mammography, ultrasonography, magnetic resonance imaging, and nuclear imaging.

Role of imaging in neoadjuvant therapy for breast cancer.

Breast lymphoma is a rare hematologic neoplasm that originates in the breast lymphoid tissue and includes primary breast lymphoma (PBL) and secondary breast lymphoma (SBL) PBL involves the breast lymphoid tissue in the absence of previously identified extramammary lymphoma and widespread disease SBL is the most common metastasis to the breast, accounting for 17% of metastatic disease to the breast PBL and SBL usually demonstrate imaging phenotypes that overlap with those of primary breast carcinoma, which makes a prospective diagnosis of breast lymphoma challenging These nonspecific imaging features include an iso- to hyperdense oval mass or masses at mammography, a hypoechoic or mixed-echogenicity hypervascular mass at US, an enhancing mass with type II kinetics at MRI, and high fluorine 18-fluorodeoxyglucose avidity at PET In cases of suspected lymphoma, reviewing the clinical history, using appropriate biopsy techniques, and evaluating for multiplicity, bilaterality, and distant disease are critical for diagnosis and management A patient with PBL generally has an earlier clinical presentation with a palpable abnormality and a solitary imaging finding In contrast, multiple masses in an older patient and an occult clinical presentation favor an SBL diagnosis ©RSNA, 2019

https://pubs.rsna.org/doi/pdf/10.1148/rg.2019180097

Primary and Secondary Breast Lymphoma: Clinical, Pathologic, and Multimodality Imaging Review.

Benign Causes of Architectural Distortion Radial Scars and Complex Sclerosing Lesions Both radial scars and complex sclerosing lesions result from idiopathic processes unrelated to trauma or postsurgical change. They are characterized microscopically by radiating ducts and lobules that show varying degrees of hyperplasia, adenosis, ectasia, or papillomatosis. Complex sclerosing lesions are usually larger than 1 cm and display more proliferative tissue than radial scars. Because most patients are asymptomatic and the lesions are not palpable, radial scars and complex sclerosing lesions are usually discovered as incidental findings on screening mammography. However, they pose a diagnostic challenge because they closely mimic scirrhous carcinomas. The typical mammographic appearance is a radiolucent central core with spiculated radiations, at times associated with microcalcifications (Fig. 1). On ultrasound, radial scars commonly present as a hypoechoic mass or parenchymal distortion that mimics malignancy. On MRI, the morphologic features and contrast enhancement patterns of radial scars and complex sclerosing lesions cannot reliably differentiate a benign from malignant process. There is only limited evidence supporting the rate of enhancement as a useful differentiating factor. Because of the inability to reliably differentiate radial scars and complex sclerosing lesions from carcinoma, the lesion must be biopsied and excised.

https://www.ajronline.org/doi/pdf/10.2214/AJR.12.10153

Vandana Dialani

Papers

Ectopic pregnancy: a review.

Increased Cancer Detection Rate and Variations in the Recall Rate Resulting from Implementation of 3D Digital Breast Tomosynthesis into a Population-based Screening Program.

Role of imaging in neoadjuvant therapy for breast cancer.

Primary and Secondary Breast Lymphoma: Clinical, Pathologic, and Multimodality Imaging Review.

Architectural Distortion of the Breast