Convolutional Neural Networks for the Automated Segmentation of Malignant Pleural Mesothelioma: Analysis of Performance Based On Probability Map Threshold
M Shenouda1*, E Gudmundsson2, F Li1, C Straus1, H Kindler1, A Dudek3, T Stinchcombe4, X Wang4, A Starkey1, S Armato1, (1) The University of Chicago, Chicago, IL, (2) UCL Hospitals NHS Trust, UK, (3) Health Partners Institute, HealthPartners Cancer Care Center, St. Paul, MN, (4) Duke University, Durham, NC
Presentations
PO-GePV-M-35 (Sunday, 7/10/2022) [Eastern Time (GMT-4)]
ePoster Forums
Purpose: To determine whether an optimal threshold exists when applied to probability maps output by a convolutional neural network (CNN) trained to segment malignant pleural mesothelioma (MPM) tumors on thoracic CT scans.
Methods: 88 baseline and follow-up CT scans of 21 patients with MPM from the Cancer and Leukemia Group B (CALGB) 30901 study were segmented by a VGG16/U-Net deep CNN architecture. Tumor contours were annotated by the CNN based on its probability maps. A radiologist modified the contours generated at a 0.5 probability threshold. Percent difference of tumor volume and overlap using the Dice Similarity Coefficient (DSC) were compared between the standard reference provided by the radiologist and the CNN’s outputs. These figures of merit were calculated for thresholds ranging between 0.001 to 0.9, investigating whether an “optimal” threshold can maximize the DSC and minimize the percent difference.
Results: Tumor volumes calculated using CNN annotations were consistently smaller than the radiologist’s contours. Absolute percent difference decreased on average, from 43.96% to 24.18%, when decreasing the probability threshold from 0.5 to 0.1. Median and mean DSC were within 0.57-0.59. DSC had a peak at 0.5 for an optimal threshold, while percent difference had no clear threshold. Because the acquired CT scans incorporated patients with varying disease morphology, visual inspection of the CNN’s contours showed it struggle with certain presentations, such as severe pleural effusion or disease in the pleural fissure.
Conclusion: This pilot study investigated the different metrics used in MPM volumetry and whether they can be used to identify an optimal threshold to be applied to a CNN probability map. While no single threshold was found for both metrics, this work highlighted the importance of considering the difference of tumor volume and overlap of tumor regions between the CNN and the standard reference when optimizing deep learning-based tumor segmentations.
Funding Support, Disclosures, and Conflict of Interest: SGA receives royalties and licensing fees for computer-aided diagnosis technology through the University of Chicago.
Keywords
Computer Vision, CT, Segmentation
Taxonomy
IM- CT: Machine learning, computer vision
Contact Email