AI in Pathology: What could possibly go wrong?

<h3>Introduction</h3> CT angiography (CTA), the modality of choice for detection of intracranial aneurysms (IA), is widely available at all care levels. Implementing deep learning algorithms (DLA) in the radiologists’ workflow could result in higher detection rates, particularly in smaller centers. We present results of multicenter pre-clinical experience with a new IA detection DLA. <h3>Methods</h3> A DLA was developed at Site 1 to predict the existence and locations of IAs &gt;2 mm in 3D head CTA volumes. It localizes IAs using an improved 3D UNet to regress the bounding boxes of suspicious locations. IA presence is confirmed by a multi-resolution 3D DenseNet at each suspicious location. The model was trained on 436 studies and tested on 696 studies from site 1. To validate robustness against protocol variability, it was retrospectively tested on 337 volumes (149 positives, 188 negatives) from 4 additional sites with various types of scanners. A neuroradiologist/trained fellow from each site annotated bounding boxes around the IAs with 3DSlicer. The model’s performance was assessed by comparing its predicted aneurysm locations to the annotated boxes. <h3>Results</h3> The model achieved a lesion-level sensitivity of 0.72 (95% CI: 0.66 - 0.75) with 1.4 false positives per volume (FPPV, 95% CI: 1.22 - 1.57). Sensitivity ranged 0.84 (1.72 FPPV) - 0.67 (1.40 FPPV) amongst 5 sites. Performance variation was secondary to testing on images from scanners the model had not been trained on (unseen). Removing unseen scanners, the model achieved sensitivity of 0.77 (95% CI: 0.68 - 0.84) with 1.51 FPPV (95% CI: 1.24 - 1.82). Patient-level area under curves (AUCs) on the external sites were comparable to that of site 1 dataset, which was 0.88 (95% CI: 0.84 - 0.92). The model performed better in the detection of internal carotid and anterior communicating artery IAs &gt; 3 mm. <h3>Conclusions</h3> This DLA for IA detection showed robust performance in detecting IAs &gt; 3 mm. DLAs may reliably help detection of IAs on CTA, particularly in the community practices less familiar with intracranial vascular imaging. Different scanners can lead to performance variability. This can be avoided by training with data from different scanners. <h3>Disclosures</h3> <b>D. Wu:</b> 1; C; National Institute of Health. <b>E. Orru’:</b> None. <b>S. Ferraciolli:</b> None. <b>A. Ashok:</b> 1; C; NIHR Academic Clinical Fellowship. <b>A. Sorby-Adams:</b> None. <b>M. Guindy:</b> None. <b>D. Montes:</b> None. <b>A. Medina:</b> None. <b>G. Bianco:</b> None. <b>J. Kaggie:</b> 1; C; NIHR Cambridge BRC. <b>D. Rabina:</b> None. <b>O. Ashush:</b> None. <b>X. Han:</b> None. <b>Y. Baror:</b> None. <b>M. Patel:</b> 4; C; Rhino HealthTech Inc. <b>I. Dayan:</b> 4; C; Rhino HealthTech Inc.. <b>J. Romero:</b> None. <b>R. González:</b> 1; C; National Institute of Health. 2; C; MR Access, Inc.. 4; C; MR Access, Inc.. <b>C. Wald:</b> None. <b>F. Kitamura:</b> 2; C; MD.ai, GE Healthcare. <b>P. Kuriki:</b> None. <b>T. Matys:</b> 1; C; NIHR Cambridge BRC. 2; C; NHS. <b>Q. Li:</b> 1; C; National Institute of Health. 

/pdf/e-107-multi-center-study-of-a-deep-learning-model-for-eqbin8mv.pdf

Q. Li

Papers

E-107 Multi-center study of a deep learning model for intracranial aneurysm detection in computed tomography angiography