A Clinical and Time Savings Evaluation of a Commercial Deep Learning Automatic Contouring Algorithm
J Ginn1*, H Gay2, J Hilliard2, J Shah3, N Mistry3, C Mohler3, G Hugo2, Y Hao2, (1) Duke University, Durham, NC, (2) Washington University School of Medicine, St. Louis, MO (3) Siemens Healthineers, Durham, NC
Presentations
PO-GePV-M-202 (Sunday, 7/10/2022) [Eastern Time (GMT-4)]
ePoster Forums
Purpose: Automatic contouring algorithms may reduce normal organ-at-risk (OAR) contouring time. However, a thorough evaluation of automatic contouring algorithms is required prior to clinical use. Here we report a quantitative, qualitative, and time saving evaluation of a prototype deep learning segmentation algorithm developed by Siemens Healthineers.
Methods: Normal pelvic and head and neck contours were evaluated retrospectively for 100 patient cases in a board-approved study. Contours generated by the software were compared to clinical contours quantitatively using the Sorensen-Dice coefficient (Dice), Jaccard index (JC), and Hausdorff distance (Haus). Performance outliers were further investigated visually to identify contouring discrepancies. To quantify the time savings, a certified medical dosimetrist manually contoured and, separately, edited the automatically generated OARs for 10 head and neck and 10 pelvic patients. The automatic, edited, and manually generated contours were visually evaluated and scored by a practicing radiation oncologist on a scale of 1-4 where a higher score indicated better performance.
Results: The quantitative comparison revealed high Dice and JC performance (>0.8) for relatively large organs such as the lungs, brain, femurs, and kidneys. Smaller structures that had relatively low Dice and JC values tended to have low Hausdorff distances. Poor performing outlier cases revealed common anatomical inconsistencies including overestimation of the bladder and incorrect superior-inferior truncation of the spinal cord and femur contours. In all cases editing contours was faster than manual contouring with an average time savings of 43.4% or 11.8 min per patient. The physician scored 240 structures resulting in an average and standard deviation score of 3.65 ± 0.59 where only 11 structures required major revision or needed to be redone.
Conclusion: Our results indicate the software has the potential to reduce clinical contouring time. The algorithm’s performance is promising but contours still require human review and some editing prior to clinical use.
Funding Support, Disclosures, and Conflict of Interest: This work was funded in part by Siemens Healthineers.
Keywords
CT, Segmentation, Radiation Therapy
Taxonomy
Contact Email