S Elguindi*, A Li, M Zhu, L Cervino, H Veeraraghavan, J Jiang, E LoCastro, Memorial Sloan Kettering Cancer Center, New York, NY

• 
  S Elguindi

SU-E-BRB-2 (Sunday, 7/10/2022) 1:00 PM - 2:00 PM [Eastern Time (GMT-4)]

Ballroom B

Purpose: Tracking AI auto segmentation usage in the clinical setting to effectively update models based on user edits and identify model deficiencies is labor intensive due to the large data requirements. We developed an automated contour edit tracking system, Voxel-Wise, to verify our AI contouring models for head-and-neck and thoracic sites.

Methods: Image data from our in-house AI contouring system is automatically pushed to our clinically approved image repository database, XNAT. Voxel-Wise starts by sorting data by anatomical site using incoming DICOM, and automated scripts monitor the clinical contouring system for updates to RTSTRUCT’s to be retrieved by XNAT. An algorithm based on user roles (planner, physicist, physician) and the sequential nature of the contouring process was applied to tag contours in three categories: initial-contour, physician-approved, and planner-approved. Using a Django web app deployed on a Kubernetes cluster, tagged data from XNAT is parsed, compared, and exposed through a REST API and GUI for fast review. Statistical process control on surface dice was done to detect outliers or uncontrolled variation.

Results: Voxel-Wise has been monitoring our AI contouring system since October 2021. A total of 1,515 AI segmentation jobs were collected resulting in 802 contouring sessions and 3,737 individual contours between the two sites. Lower Control Limits for the 16 AI-based contours were set based on the first month’s data. 61 contours, representing 1.6% of data analyzed, were detected as outliers in terms of the amount of editing required to make it clinically acceptable and no uncontrolled variations were observed.

Conclusion: We demonstrate the practical use of an automated system to track user edits of AI auto-segmentation. Using this system, we are able to at a glance look for unusual variations due to unforeseen process changes and have the ability to practically implement forcing function for continual model improvement.

Keywords

Computer Software, Segmentation

Taxonomy

IM/TH- image Segmentation: CT

Contact Email