Attention-Based Multiple Instance Learning AI in the Prediction of Treatment for COVID-19 Patients
J Fuhrman1*, C Wei2,3, B Katsnelson1, E Katsnelson1, H Li1, Z Luo4, Z Dong5, F Lure6, Z Cheng2,3, M Giger1, (1) Department of Radiology, The University of Chicago, Chicago, IL, (2) Department of Critical Care Medicine, Zhongnan Hospital Of Wuhan University, Wuhan, China (3) Wuhan Research Center for Infectious Diseases and Cancer, Chinese Academy of Medical Sciences, Wuhan, China (4) Department of Critical Care Medicine, Fudan University, Zhongshan Hospital, Shanghai, China (5) Suzhou Zhizhun Medical Technology Co. Ltd., Suzhou, China (6) MS Technologies Corp, Rockville, MD
Presentations
PO-GePV-M-40 (Sunday, 7/10/2022) [Eastern Time (GMT-4)]
ePoster Forums
Purpose: To utilize attention-based multiple instance learning (AMIL) with transfer learning on CT scans for the prediction of steroid administrations for hospitalized COVID-19 patients.
Methods: This retrospective study utilized de-identified CT scans acquired on 864 confirmed COVID-19 patients, hospitalized between November 14, 2019 and September 8, 2020. An AMIL scheme extracted normalized CT slice level feature representations from the maximum pooling layers of a VGG19 architecture pre-trained for ImageNet classification. Slice representations were pooled to form CT scan level representations through learned attention weights that identify influential slices within the CT scan. Note that the attention weights can be used as interpretable output for clinical review by the radiologist as well as validation of model performance. Next a support vector machine was trained to predict if a patient would receive steroid administrations during their course of treatment with classification performance evaluated through receiver operating characteristic (ROC) analysis with the area under the curve (AUC) serving as the figure of merit. This is a weakly supervised classification problem with noisy labeling due to highly variable time interval between image acquisition and subsequent steroid administration. Uniform manifold approximation projection (UMAP) was used to visualize scan representations to better understand performance and evaluate potential bias based on CT scanner manufacturer and patient age. Confidence intervals were determined across 5-fold cross validation (by patient) with data divided into training/validation/testing subsets of 70%/10%/20% of the data.
Results: The AMIL transfer learning algorithm yielded an AUC of 0.80 +/- 0.05 in the task of predicting which patients received steroids regardless of the time between administration and image acquisition. UMAP visualization did not show any bias due to age or scanner manufacturer.
Conclusion: The AMIL approach demonstrated strong potential for the difficult clinical task of predicting treatment with steroids for COVID-19 patients.
Funding Support, Disclosures, and Conflict of Interest: Supported by the non-profit Central Research Institute Fund of Chinese Academy of Medical Sciences 2020-PT320-004. Partial funding was provided by NIH S10 OD025081 and NIBIB COVID-19 Contract 75N92020D00021. MLG receives royalties from Hologic, GE Medical Systems, MEDIAN Technologies, Riverain Medical, Mitsubishi and Toshiba, and was a cofounder in Qlarity Imaging.
Keywords
Not Applicable / None Entered.
Taxonomy
Not Applicable / None Entered.
Contact Email