Multi-Institutional Validation of a Convolutional Neural Network-Based Approach to the Detection of Vertebral Body Misalignments in Planar X-Ray Setup Images
J Charters1, J Kunz2, J Lamb1, D Low1, O Morin3, R Petragallo1*, D Saenz4, B Salter2, G Valdes3, B Ziemer3, (1) University of California Los Angeles, Los Angeles, CA, (2) University of Utah, Huntsman Cancer Hospital, Salt Lake City, UT, (3) University of California San Francisco, San Francisco, CA, (4) University of Texas HSC SA, San Antonio, TX
Presentations
SU-D-TRACK 6-4 (Sunday, 7/25/2021) 2:00 PM - 3:00 PM [Eastern Time (GMT-4)]
Purpose: Alignment to the incorrect vertebral body remains a rare but serious patient safety risk in image-guided radiotherapy (IGRT). Our group has previously presented a prototype convolutional neural network (CNN)-based approach for automatic detection of these errors in a planar stereoscopic x-ray IGRT system. Here we expand on our previous work to examine the robustness of our model when applied to multi-institutional data.
Methods: A single-institutional model was trained using x-ray and digitally reconstructed radiograph (DRR) image pairs from 87 thoracic spine radiotherapy patients treated at a single institution using a stereoscopic x-ray IGRT system. Training data used simulated off-by-one vertebral body errors created by shifting DRRs by one vertebral body in both directions along the spinal column using a semi-automated method. 70 patient datasets were used to train the CNN model and 17 used for testing the detection accuracy. This trained model was then applied to 167 patient datasets collected from 3 external institutions to evaluate model robustness. Subsequently, the CNN was re-trained using data from all four institutions. Of this combined dataset, 204 patients were used for re-training and 49 reserved for testing. For all datasets, all treatment fractions were used in each of the patients.
Results: When the single-institution CNN was used to classify unseen image pairs from collaborating institutions, the resulting receiver operating characteristic area under the curve (AUC) was 0.94. However, with the specificity fixed at 99% the corresponding sensitivity was 32.9%. For the model re-trained and tested on multi-institutional data, an AUC of 0.99 was obtained, and a sensitivity of 91.9% when specificity was fixed at 99%.
Conclusion: We have demonstrated a CNN-based vertebral body misalignment model that is robust to multi-institutional image data. The incorporation of cross-institutional data into the training dataset significantly improves final classification accuracy.
Funding Support, Disclosures, and Conflict of Interest: This work supported in part by AHRQ R01 HS026486.
Handouts
Keywords
Not Applicable / None Entered.
Taxonomy
Not Applicable / None Entered.
Contact Email