Clinically Applicable Deep Learning Strategy for Pulmonary Nodule Risk Prediction: Insights into HONORS

TL;DR: The combination of HONORS and FGP-NET provides well-organized stratification for pulmonary nodules and also offers the potential for reducing medical errors.

Abstract: Background and Purpose Limited optimization was clinically applicable for reducing missed diagnosis, misdiagnosis and inter-reader variability in pulmonary nodule diagnosis. We aimed to propose a deep learning-based algorithm and a practical strategy to better stratify the risk of pulmonary nodules, thus reducing medical errors and optimizing the clinical workflow. Materials and Methods A total of 2,348 pulmonary nodules (1,215 with lung cancer) containing screened nodules from National Lung Cancer Screening Trial (NLST) and incidentally detected nodules from Jinling Hospital (JLH) were used to train and evaluate a deep learning algorithm, Filter-guided pyramid network (FGP-NET). Internal and external test of FGP-NET were performed on two independent datasets (n=542). The performance of FGP-NET at Youden point which maximizing the Youden index was compared with 126 board-certificated radiologists. We further proposed Hierarchical Ordered Network ORiented Strategy (HONORS), which manipulates the emphasis either on sensitivity or specificity to target risk-stratified clinical scenarios, directly making decisions for some patients. Results FGP-NET achieved a high area under the curve (AUC) of 0.969 and 0.855 for internal and external testing, and was comparable or even outperformed the radiologists when considering sensitivity. HONORS-guided FGP-NET identified benign nodules with a high sensitivity (95.5%) in the screening scenario, and demonstrated satisfactory performance for the rest ambiguous nodules with 0.945 of AUC by the Youden point. FGP-NET also detected lung cancer with a high specificity of 94.5% in routine diagnostic scenario; an AUC of 0.809 was achieved for the rest nodules. Conclusion The combination of HONORS and FGP-NET provides well-organized stratification for pulmonary nodules and also offers the potential for reducing medical errors. Highlights Pulmonary nodules were managed for both screening and diagnostic scenarios Proposal of a hierarchical strategy for targeting risk-stratified clinical scenarios A large scale Human-deep learning contest for reliable performance evaluation

Summary

Introduction

• Lung cancer is one of the commonest cancers and the leading cause of cancer mortality worldwide 1,2.
• Every step from screening to final diagnosis is indispensable which makes the entire process time- and labor-consuming.
• The desire to improve the efficacy and efficiency of clinical care continues to drive multiple innovations into practice, including deep learning (DL).
• Most of previous works focused on the screening population, and studies based on incidentally detected nodules in routine diagnostic scenario with higher risk were limited.
• The benign nodules or lung cancer can be accurately identified in screening and routine diagnostic scenarios in step-1 and further stratification of ambiguous nodules was performed to aid clinical decision making in step-2.

Datasets

• Ethical approval was obtained for this retrospective study, and informed consent was waived for reviewing patients’ medical records.
• The authors retrospectively analyzed 16,801 patients who underwent surgery or biopsy due to lung lesions from May 2009 to June 2018 in Jinling hospital.
• The authors analyzed 1,060 patients confirmed as having lung cancer and 5,275 patients as not in the NLST.
• Details for the data curation are available in the Supplementary Information, Inclusion and exclusion criteria.

Annotation and preprocessing

• Automatic nodule detection was performed using the Dr. wise platform 17 (the detection network could be found in the supplementary information, Pulmonary nodule detection network) and the geometric centers of the enrolled nodules were further revised by two radiologists.
• Due to the differences in pixel spacing and slice thickness, the CT images were subsequently linearly interpolated into 3D isotropic images with voxel spacing of 0.6 × 0.6 × 0.6 mm3.
• Since the datasets were relatively small compared to traditional image classification datasets for deep learning such as cifar1018 and ImageNet19, the authors conducted heavy data augmentations on all initially generated image patches (a size of 128 × 128 × 128) containing the nodules, e.g., 0-360 degree of random rotation, random zooming in or zooming out, random cropping and flipping.

Development and training of FGP-NET

• JLH and NLST dataset were randomly assigned into one of the following three sets: training set (JLH, 1086 nodules; NLST, 520 nodules) for optimizing network weights, validation set (JLH, 100 nodules; NLST, 100 nodules) for deciding the values of hyperparameters and internal test set (JLH, 100 nodules; NLST, 200 nodules) for evaluating the performance.
• The patients in the training, validation and test sets were exclusive to each of the other data sets.
• To this end, the authors aimed to capture both local and global features and their interactive relationships to better represent the nodule.
• By concatenating the clear attention map distilled by local feature extractors and raw feature maps from early stage of network, FGP-Net was able to keep high resolution details to describe small-sized local features and to use the accurate localization of large ones to guide small ones.
• DenseNet was one of the state-of-art network structures in computer vision.

Validation of FGP-NET

• FGP-Net generated continuous numbers between 0 and 1 for nodule risk stratification, being consistent with the malignancy probability of the nodules, named ‘malignancy score’.
• The corresponding sets were defined as S1, S2, S3 and S4, respectively.
• To keep up with the input of FGP-NET, radiologists were only informed with the precise location of the nodules while blinded to the medical history and pathological results.
• The performance of radiologists was evaluated at the average level and majority level (voting the scores rated by 126 radiologists) when compared with FGP-NET.
• All statistical tests used in this study were 2-sided and a P-value less than .05 was considered significant.

Interpretation of learned features

• Given the black box property of DL, the authors further conducted a two-way feature interpretation to explore whether FGP-NET learned solid and effective features.
• More specifically, T-distributed Stochastic Neighborhood Embedding (t-SNE) 24 and probability heat-map were applied for visualization of global features and local features (Supplementary Information, Feature visualization of FGP-Net).

Evaluation of HONORS

• To validate the performance of HONORS in screening and routine diagnostic scenarios, the authors simulated the its application on NLST test set and multi-center set.
• The precision of the stratified nodules was evaluated using negative predictive value (NPV) and positive predictive value (PPV), respectively.
• The authors further compared the performance between HONORS and 126 radiologists in the incidentally detected nodules using JLH test set (Supplementary information, Comparison of HONORS with radiologists).
• In addition to the two-step way targeted on different scenarios, HONORS can also be realized in a three-step way regardless of scenarios .
• In the first step, FPG-NET at HSen point stratified the benign nodules; in the second step, FPG-NET at HSpe point stratified the lung cancer; and in the third step, FPG-NET at Youden point stratified the rest ambiguous nodules.

Results

• Overview of the study design and results A total of 2,348 pulmonary nodules (1,215 malignant nodules) containing screened and incidentally detected nodules found by chest CT were used to train and evaluate their DL algorithm, FGP-NET .
• The authors further investigated whether the FGP-NET was comparable or even superior to radiologists.
• It was still at a relatively low level even in consultant group (κw < 0.4).
• A total of 18 nodules was misdiagnosed by radiologists’ majority opinion and FGP-NET inconsistently, accounting for 9 each .

Interpretation of Features Learned by FGP-NET

• DL is frequently referred to as a black box—data goes in, decisions come out, but the processes between input and output are opaque 25.
• It is crucial to enable the black box to be opened, and thus, the authors interpreted the global features by using t-SNE which is particularly well suited for the visualization of high-dimensional data.
• To be specific, these attribution regions were mainly located in benign features for those benign nodules.
• Nevertheless, to malignant nodules, regions were predominately situated in malignant features, suggesting that FGP-NET may concentrate on the irregular margin of the nodule and solid component within part-solid nodules .
• These attribution regions were certainly consistent with the visual observation by radiologists.

Proposal of HONORS

• This does not eliminate the role of radiologists in making the final decision.
• Therefore, the authors further proposed HONORS, a novel two-step hierarchical strategy for clinical application of FGP-NET.
• HONORS was applied to NLST test set and multi-center set to appraise its performance.
• Additionally, a three-step way to realize HONORS was also evaluated in this context .

Discussion

• The authors developed and validated a DL algorithm—FGP-NET that is capable of stratifying pulmonary nodules with great performance and is comparable with a large group of 10 / 20 radiologists.
• The authors study was conceptually practical in clinics because FGP-NET and HONORS were designed to tailor both screening and routine diagnostic scenarios.
• Pyramid structure that the authors harnessed to support FGP-NET is a method for extracting multi-scale features and is quite suitable for feature extraction of pulmonary nodules due to its large size variation 26.
• Huang et al. developed a computer-aided diagnosis approach with a sensitivity of 95% and a specificity of 88% which outperformed three radiologists’ combined reading using 186 nodules from NLST dataset 29.
• 12 / 20 Taken together, the authors proposed HONORS to lay groundwork toward application of the DL-based pulmonary nodule stratification algorithm in the screening and routine diagnostic scenarios.

Figures

Figure 1: The framework of the two-step hierarchical strategy and the overall flow chart of this study. a) We proposed HONORS, a hierarchical strategy for pulmonary nodules risk stratification in different clinical scenarios, involves three operating points of FGP-NET, as well as a two-step method. The first step includes the blue or yellow translucent paths that focus on either sensitivity or specificity for screening or routine diagnosis scenarios, resulting in a portion of nodules of interest with high precision being detected without any intervention of physicians (radiologists) — “Human Free”. Further, under a Youden point in

Figure 4：Performance of HONORS in screening scenario using NLST test set. a) In step 1, each line represents the relative malignant score (y-axis) of one nodule and x-axis represents the nodule index. Nodules with scores lower than 0.000005 were directly considered as benign nodules. The remaining ambiguous nodules will flow in to b) step 2 where each line represents the corresponding malignant score that extracted by Youden point of 0.384642. Nodules with scores less than zero were predicted as benign nodules and malignant ones for the rest. c) Corresponding statistics were used to evaluate the performance of HONORS in screening scenario, including AUC, sensitivity, specificity and NPV.

Figure 5: Performance of HONORS in routine diagnostic scenario using multi-center set. In step 1, each line represents the relative malignant score (y-axis) of one nodule in a) and x-axis represents the nodule index, and nodules with scores greater than 0.999965 were directly considered as malignant nodules. The remaining ambiguous nodules will flow into b) step 2 where each line represents the corresponding malignant score that extracted by Youden point of 0.384642. Nodules with scores greater than zero were predicted as malignant nodules and benign ones for the rest. c) Corresponding statistics were used to evaluate the performance of HONORS under routine diagnostic scenario, including AUC, sensitivity, specificity and PPV.

Figure 3: Opening the black box of FGP-NET by both t-SNE and probability heat-maps. a) t-SNE visualization of features extracted from the last fully-connected layer of the global branch of FGP-NET. Global image features learned by FGP-NET were visualized as a 2-dimensional scatter plot, with each point corresponding to a nodule that derives from either JLH, NLST or Multi-center dataset. Heat-maps of sample images reveal the regions that most positively influence the prediction of FGP-NET. The left column represents the central axial slice of the 48 × 48 patch network input. The right column represents the overlaid activation region, with red and blue showing the highest and lowest contributions, respectively. In benign nodules, activated maps were mainly centrally located in benign features (b-f). b-c) Granuloma, the activated region was in the middle of the satellite lesions. d-e) Hamartoma, the activated region was in the center of nodule. f) Inflammatory pseudotumor, the activated region was long speculation. In malignant nodules, they were mainly off-center located in malignant features (g-k).

Table 1: Performance of FGP-NET during internal and external test

Figure 2: Performance comparison between FGP-NET and radiologists. a) We conducted a comparison study with a group of 126 radiologists. They reviewed and rated the nodules from 1 to 4 (4 denotes the highest malignancy). Performances of FGP-NET (black line) versus all radiologists (lake blue, yellow and pink points) are presented. Each point represents a combination of sensitivity and specificity for a radiologist. The indigo blue point represents the average performance of the radiologists, with error bars denoting the SD. The red cross mark represents the sensitivity and specificity of radiologist’ majority opinion (voting the score rated by 126 radiologists). b) The inter-observer agreement among label, FGP-NET, majority of radiologists and each radiologist. Inter-observer agreement between 126 radiologists was fair (κw = 0.315 ± 0.139). Although the inter-observer agreement grew with increased experience, inter-observer variability was huge even in consultants in pulmonary nodule diagnosis. c) Correlation analyses between the sensitivity, specificity and the rating time of each radiologist are shown.

Full text

1 / 20
Clinically Applicable Deep Learning Strategy for Pulmonary Nodule Risk
Prediction: Insights into HONORS
Wenhui Lv
1,2,13
, Yang Wang
3,13
, Changsheng Zhou
2,13
, Sheng Huang
4
, Xiangming Fang
5
,
Qiuzhen Xu
6
, Qirui Zhang
1,2
, Chuxi Huang
2
, Xinyu Li
1,2
, Zhen Zhou
7
, Yizhou Yu
8
,Yizhou
Wang
7
, Mengjie Lu
2
, Qiang Xu
2
, Xiuli Li
9
, Haoliang Lin
9
, Xiaofan Lu
10
, Qinmei Xu
2
, Jing
Sun
2
,Yuxia Tang
2
, Yong Song
11
, Fangrong Yan
10
, Bing Zhang
3
, Zhen Cheng
12
, Longjiang
Zhang
1,2
, Guangming Lu
1,2*
Author Affiliations:
1
Department of Medical Imaging, Jinling Hospital, Southern Medical University, Nanjing,
China
2
Department of Medical Imaging, Jinling Hospital, Nanjing University School of Medicine,
Nanjing, China
3
Department of Radiology, the Affiliated Nanjing Drum Tower Hospital of Nanjing
University Medical School, Nanjing, China
4
Department of Radiology, Affiliated Hospital 2 of Nantong University, Nantong, China
5
Department of Medical Imaging, Wuxi People’s Hospital, Wuxi, China
6
Department of Radiology, Southeast University Zhongda Hospital, Nanjing, China
7
Computer Science Department, School of EECS, Peking University, Beijing, China
8
University of Hongkong, Hongkong, China
9
Deepwise AI Lab, Deepwise Inc., Beijing, China
10
Research Center of Biostatistics and Computational Pharmacy, China Pharmaceutical
University, Nanjing, China
11
Department of Respiratory Medicine, Jinling Hospital, Nanjing University School of
Medicine, Nanjing, Jiangsu, China
12
Department of Radiology, Stanford University, School of Medicine, Stanford, CA, United
States,
13
These authors contributed equally
*Corresponding author: Guangming Lu, MD, Department of Medical Imaging, Nanjing
Jinling Hospital, Nanjing University School of Medicine, 305#, Eastern Zhongshan Rd,
Nanjing 210002, China (Email: cjr.luguangming@vip.163.com)
All rights reserved. No reuse allowed without permission.
perpetuity.
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in
The copyright holder for thisthis version posted February 7, 2020. ; https://doi.org/10.1101/2020.02.03.20020297doi: medRxiv preprint
NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.

2 / 20
Abstract
Background and Purpose: Limited optimization was clinically applicable for reducing
missed diagnosis, misdiagnosis and inter-reader variability in pulmonary nodule diagnosis.
We aimed to propose a deep learning-based algorithm and a practical strategy to better stratify
the risk of pulmonary nodules, thus reducing medical errors and optimizing the clinical
workflow.
Materials and Methods: A total of 2,348 pulmonary nodules (1,215 with lung cancer)
containing screened nodules from National Lung Cancer Screening Trial (NLST) and
incidentally detected nodules from Jinling Hospital (JLH) were used to train and evaluate a
deep learning algorithm, Filter-guided pyramid network (FGP-NET). Internal and external
test of FGP-NET were performed on two independent datasets (n=542). The performance of
FGP-NET at Youden point which maximizing the Youden index was compared with 126
board-certificated radiologists. We further proposed Hierarchical Ordered Network ORiented
Strategy (HONORS), which manipulates the emphasis either on sensitivity or specificity to
target risk-stratified clinical scenarios, directly making decisions for some patients.
Results: FGP-NET achieved a high area under the curve (AUC) of 0.969 and 0.855 for
internal and external testing, and was comparable or even outperformed the radiologists when
considering sensitivity. HONORS-guided FGP-NET identified benign nodules with a high
sensitivity (95.5%) in the screening scenario, and demonstrated satisfactory performance for
the rest ambiguous nodules with 0.945 of AUC by the Youden point. FGP-NET also detected
lung cancer with a high specificity of 94.5% in routine diagnostic scenario; an AUC of 0.809
was achieved for the rest nodules.
Conclusion: The combination of HONORS and FGP-NET provides well-organized
stratification for pulmonary nodules and also offers the potential for reducing medical errors.
Keywords: clinically applicable hierarchical strategy; Filter-guided pyramid network;
pulmonary nodule; human-AI contest; computed tomography
Highlights
• Pulmonary nodules were managed for both screening and diagnostic scenarios
• Proposal of a hierarchical strategy for targeting risk-stratified clinical scenarios
• A large scale Human-deep learning contest for reliable performance evaluation
All rights reserved. No reuse allowed without permission.
perpetuity.
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in
The copyright holder for thisthis version posted February 7, 2020. ; https://doi.org/10.1101/2020.02.03.20020297doi: medRxiv preprint

3 / 20
Introduction
Lung cancer is one of the commonest cancers and the leading cause of cancer mortality
worldwide
1,2
. Survival rate is hugely influenced by stage at diagnosis and early diagnosis is
full of challenges
3
. Every step from screening to final diagnosis is indispensable which
makes the entire process time- and labor-consuming. For better management and diagnosis of
pulmonary nodules, Lung-RADs guidelines
4
, recommendations from Fleschnier society
5
have been proposed. However, the lack of awareness of these guidelines and the inter-reader
variability have limited the broader use of them
6,7
. Besides, the detection of pulmonary
nodules is rising up dramatically and physicians are experiencing burnout at increasing rate.
Excessive workload is resulting in diagnostic errors including missed diagnosis, and
misdiagnosis
8
. Therefore, there is an unmet need for relieving working pressure of physicians
and reducing the incidence of such medical errors. To the best of our knowledge, limited
literature has delved into this place for a clinically applicable optimization.
The desire to improve the efficacy and efficiency of clinical care continues to drive multiple
innovations into practice, including deep learning (DL). In recent, DL has infiltrated the
optimization and streamlining of clinical workflows, quietly improving, changing and
reconstructing the way health care works
9-13
, especially for pulmonary nodules management
12,14
. However, most of previous works focused on the screening population, and studies based
on incidentally detected nodules in routine diagnostic scenario with higher risk were limited.
In this study, we reasoned that DL would play distinct roles in the risk-stratified clinical
scenarios for pulmonary nodule screening and diagnosis. Therefore, we proposed a clinically
applicable DL-based algorithm—Filter-guided pyramid network (FGP-NET), and a practical
strategy—Hierarchical-Ordered Network-ORiented Strategy (HONORS), which involves two
steps for two different clinical scenarios (i.e., screening and routine diagnostic scenarios)
(Figure 1a). The benign nodules or lung cancer can be accurately identified in screening and
routine diagnostic scenarios in step-1 and further stratification of ambiguous nodules was
performed to aid clinical decision making in step-2. Consequently, HONORS would directly
make decision for some patients without any human intervention and assist physicians to
better manage other ambiguous nodules. It has great potential to provide well-organized
management for pulmonary nodules by optimizing the clinical workflow and reduce medical
errors through rapid and accurate image interpretation.
Methods
Datasets
Ethical approval was obtained for this retrospective study, and informed consent was waived
for reviewing patients’ medical records.
All rights reserved. No reuse allowed without permission.
perpetuity.
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in
The copyright holder for thisthis version posted February 7, 2020. ; https://doi.org/10.1101/2020.02.03.20020297doi: medRxiv preprint

4 / 20
For training and validation of FGP-NET, we incorporated both incidentally detected and
screened nodules from Jinling Hospital (JLH) and National Lung Cancer Screening Trial
(NLST), respectively. We retrospectively analyzed 16,801 patients who underwent surgery or
biopsy due to lung lesions from May 2009 to June 2018 in Jinling hospital. Nodules were
classified as either malignant or benign according to the World Health Organization (WHO)
classification of lung tumors in 2015
15
. Finally, a total of 1,286 nodules (935 malignant
nodules and 351 benign nodules) found in 1,238 patients (591 males and 647 females; 56.48 ±
11.50 years of age [mean ± standard deviation (SD)]) were included. We analyzed 1,060
patients confirmed as having lung cancer and 5,275 patients as not in the NLST. A total of 820
nodules (93 malignant nodules and 727 benign nodules) found in 442 participants (244 males
and 198 females; 61.71 ± 5.32 years of age) were included. All participants enrolled in NLST
signed an informed consent developed and approved by the screening centers’ Institutional
Review Board (IRB), the National Cancer Institute (NCI) IRB and the Westat IRB. The data
and images were obtained from the National Cancer Institute Cancer Data Access System
16
.
We also conducted an external test of FGP-NET using a multi-center set. Multi-center set
contained consecutive 235 patients (104 males and 131 females; 58.48 ± 10.75 years of age)
with 242 incidentally detected nodules (187 malignant nodules and 55 benign nodules) from
three tertiary hospitals. These patients all underwent biopsy or surgery. Details for the data
curation are available in the Supplementary Information, Inclusion and exclusion criteria. All
patients included in this study underwent nonenhanced chest computed tomography (CT) and
the CT images were reconstructed with a thickness less than 2.5 mm (see more details of
acquisition and reconstruction parameters in Supplementary Table 1).
Annotation and preprocessing
Automatic nodule detection was performed using the Dr. wise platform
17
(the detection
network could be found in the supplementary information, Pulmonary nodule detection
network) and the geometric centers of the enrolled nodules were further revised by two
radiologists. Due to the differences in pixel spacing and slice thickness, the CT images were
subsequently linearly interpolated into 3D isotropic images with voxel spacing of 0.6 × 0.6 ×
0.6 mm3. Since the datasets were relatively small compared to traditional image classification
datasets for deep learning such as cifar10
18
and ImageNet
19
, we conducted heavy data
augmentations on all initially generated image patches (a size of 128 × 128 × 128) containing
the nodules, e.g., 0-360 degree of random rotation, random zooming in or zooming out,
random cropping and flipping. Processed patches were cropped to a size of 48 × 48 × 48 and
used for algorithm training.
All rights reserved. No reuse allowed without permission.
perpetuity.
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in
The copyright holder for thisthis version posted February 7, 2020. ; https://doi.org/10.1101/2020.02.03.20020297doi: medRxiv preprint

5 / 20
Development and training of FGP-NET
JLH and NLST dataset were randomly assigned into one of the following three sets: training
set (JLH, 1086 nodules; NLST, 520 nodules) for optimizing network weights, validation set
(JLH, 100 nodules; NLST, 100 nodules) for deciding the values of hyperparameters and
internal test set (JLH, 100 nodules; NLST, 200 nodules) for evaluating the performance. The
patients in the training, validation and test sets were exclusive to each of the other data sets.
Pulmonary nodule evaluation on chest CT is of great challenge partly due to the significant
overlap and complicated interaction among visual features. To this end, we aimed to capture
both local and global features and their interactive relationships to better represent the nodule.
We designed a new U-shape network structure named FGP-Net which could produce a feature
pyramid for different-sized local features (Figure 1b). By concatenating the clear attention
map distilled by local feature extractors and raw feature maps from early stage of network,
FGP-Net was able to keep high resolution details to describe small-sized local features and to
use the accurate localization of large ones to guide small ones. Additionally, we added one
more backbone module to dig complex semantic information and relationships between these
various local ones. We applied Densely Connected Convolutional Network (DenseNet)
20
as
our backbone network and Discriminative Filter Learning (DFL)
21
modules as local feature
extractors for details of nodules. DenseNet was one of the state-of-art network structures in
computer vision. The core of Densenet was a so-called dense block, in which every layer was
connected directly to all the subsequent layers. DFL Module was a two-stream structure,
enabling network to detect discriminative mid-level patches under weakly supervised
condition.
FGP-Net was trained by using a batch size of 4 for 400 epochs and learning rate of 0.0001,
with adam optimizer and 0.0001 weight decay. After 400 epochs, the training process was
stopped due to the convergence of both the area under the curve (AUC) and cross-entropy loss
(Supplementary Figure 1).
Validation of FGP-NET
FGP-Net generated continuous numbers between 0 and 1 for nodule risk stratification, being
consistent with the malignancy probability of the nodules, named ‘malignancy score’.
Receiver operating characteristic (ROC) curves were generated by plotting sensitivity and 1 -
specificity, and AUCs were calculated to evaluate the discrimination
22
. Three operating
points were chosen for FGP-NET on the validation set; a high sensitivity (HSen)/specificity
(HSpe) point corresponded to a sensitivity/specificity of 99% to stratify the benign/malignant
nodules with high precision and a Youden point maximized the value of the Youden index for
generic discrimination between malignant and benign nodules.
Human-DL contest between FGP-NET and 126 radiologists
All rights reserved. No reuse allowed without permission.
perpetuity.
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in
The copyright holder for thisthis version posted February 7, 2020. ; https://doi.org/10.1101/2020.02.03.20020297doi: medRxiv preprint

Frequently asked questions

Q1. What have the authors contributed in "Clinically applicable deep learning strategy for pulmonary nodule risk prediction: insights into honors" ?

Wang et al. this paper proposed a clinically applicable DL-based algorithm, Filter-guided pyramid network ( FGP-NET ), and a practical strategy, Hierarchical-Ordered Network-ORiented Strategy ( HONORS ), which involves two steps for two different clinical scenarios ( i.e., screening and routine diagnostic scenarios ). 

Q2. What future works have the authors mentioned in the paper "Clinically applicable deep learning strategy for pulmonary nodule risk prediction: insights into honors" ?

To further applied FGP-NET to different clinical settings, a novel two-step strategy—Hierarchical-Ordered Network-ORiented Strategy ( HONORS ) was proposed. Based on FGP-NET, the authors further proposed a two-step strategy—HONORS, which is promising to optimize clinical workflow and realize personalized precise treatment of pulmonary 11 / 20 nodules. Future studies are warranted to prospectively assess the performance and generalizability of the algorithm at a variety of sites in real-world-use scenarios, and determine how the HONORS would impact diagnostic accuracy and clinical workflow. It may have potential to accelerate the process of pulmonary nodule diagnosis and to free doctors, nurses and other healthcare professionals to focus on providing real care for patients.