Understanding Robustness of Deep Neural Network: A Comprehensive Robustness Analysis for a Deep Learning-Based Lung-Nodule Classification Model of CT Images with Respect to Image Noise
Presentations
PO-GePV-M-48 (Sunday, 7/25/2021) [Eastern Time (GMT-4)]
Purpose: It was found that robustness of deep neural networks (DNNs) is often a concern and the reason for DNNs being robust or not is still unclear. In this work, we performed comprehensive numerical studies to get some insights on robustness behaviors of DNNs using lung nodule classification based on CT images as a testbed.
Methods: We trained DNNs for lung nodule classification based on CT images. We added to the input of the trained DNNs realistic CT noise at 100mAs generated under a real noise power-spectrum and monitor if the output is affected. We investigated the correlation between robustness and network architecture by systematically studying a class of 16 DNNs with different numbers of layers and widths. To understand the unrobustness behavior, we visualized the extracted low-dimensional representations of the input by DNNs with/without noise added, and characterized the robustness via the distance of the representations to the decision boundary determined by the DNN in this low-dimension representation space.
Results: The networks were trained to achieve Area Under the Curve of 0.89-0.93 in independent testing data. CT noise was able to affect the prediction results of all the DNNs models with up to 21.4% predicted labels altered by noise. We did not observe a trend between robustness and network architectures. By visualizing the input data in the representation space, the unrobustness behavior was ascribed to the sometimes large perturbations to the sample representations by noise, as well as the unfavorable placement of the decision boundary formed in the training process.
Conclusion: Robustness concern existed in the class of DNNs studied here for the classification problem of interest. The reasons for not being robust were identified, which may help developing training strategies to improve model robustness.
ePosters
Keywords
Not Applicable / None Entered.
Taxonomy
IM/TH- Informatics: Informatics in Therapy (general)
Contact Email