Click here to

Session: Advancing Science to Expand Access to State-of-the-Art Applications in Medical Physics: I [Return to Session]

Progressively Refined Joint Registration-Segmentation (ProSeg) of Gastrointestinal Organs at Risk From Cone-Beam CT

J Jiang1*, J Hong2, N Tyagi1, M Reyngold1, C Crane1, H Veeraraghavan1, (1) MSKCC, New York, NY, (2) Md Anderson Cancer Center, ,,


TU-D1030-IePD-F5-1 (Tuesday, 7/12/2022) 10:30 AM - 11:00 AM [Eastern Time (GMT-4)]

Exhibit Hall | Forum 5

Purpose: Fast and reliably accurate segmentation of highly mobile and deforming mobile upper gastro-intestinal (GI) organs from treatment cone-beam CTs (CBCT) are needed for quantification of dose and for use in adaptive replaning during ablative radiation treatment for locally advanced pancreatic cancers. Hence, we developed a joint registration-segmentation method that progressively refines segmentation of GI organs using patient-specific segmentation priors from planning CT scans.

Methods: A progressively refined registration-segmentation called ProSeg using a jointly trained recurrent registration-segmentation sub-networks was developed to segment GI organs, stomach-duodenum and small bowel from CBCT. Both networks were implemented using 3D convolutional long short term memory network (CLSTM) models in their encoders to progressively improve segmentation by incrementally improving local regional alignment. The registration network was trained in an unsupervised way without any ground truth deformation vector fields using pairs of planning CT (pCT) and CBCT 3D volumes. The segmentation network was trained in a supervised way using expert CBCT segmentations using pCT, its segmentation, and CBCT volumes as inputs. Segmentation consistency and deformation smoothness losses were used to regularize the registration network. Accuracy evaluation was performed on validation folds not used in 5-fold cross-validation training with 80 pCT-CBCT pairs (1 pCT and 2 CBCT scans) from 40 patients.

Results: ProSeg was more accurate than a previously published deep learning method applied to the same data resulting in an accuracy of 0.72±0.01 DSC, 10.51±2.49 mm HD95 for small bowel and 0.76±0.03 DSC, 11.16±1.32 mm HD95 for stomach-duodenum. It took < 5secs to segment entire 3D volume.

Conclusion: ProSeg showed preliminary feasibility to reliably segment small bowel and stomach from CBCT images. Evaluation on larger clinical datasets is needed to evaluate feasibility for clinical translation.

Funding Support, Disclosures, and Conflict of Interest: Jun Hong performed this work when worked at MSKCC; travel, Crane has honoraria and research agreement with Elekta;


Not Applicable / None Entered.


Not Applicable / None Entered.

Contact Email