A Realistic DRR Rendering Method Based On Cycle-GAN
R Wei1*, B Liu2,3, B Liang1, J Dai1, (1) National Cencer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing,100021, CN, (2) Image Processing Center,Beihang University, Beijing,CN, (3) Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, CN
Presentations
SU-IePD-TRACK 3-1 (Sunday, 7/25/2021) 5:30 PM - 6:00 PM [Eastern Time (GMT-4)]
Purpose: In recent study, deep learning network trained with paired data was utilized to reduce the intensity inconsistency between DRR and x-ray projection. But for lung cancer patients, it is hard to obtain adequate paired data for network training due to respiration. In this work, we developed and validated a DRR rendering method based on Cycle Generative Adversarial Networks (Cycle-GAN), which could improve the similarity between DRR and x-ray projection using unpaired training data.
Methods: In the proposed method, a Cycle-GAN was trained to generate synthetic x-ray projection from DRR, which was used to resemble real x-ray projection in tumor localization algorithms. Since respiratory motion made it hard to obtain paired data of DRR and x-ray projection, the Cycle-GAN was trained with unpaired data of DRR and x-ray projection from multiple patients. Wasserstein distance and gradient penalty terms were added to the loss function to improve the performance of the network. To apply to the new patient, the synthetic x-ray projection can be generated with DRR fed into the trained network.
Results: The proposed method was evaluated with four patients’ data. Three patients’ data was utilized to train the Cycle-GAN, and the last one was used to test the network. The x-ray projections at angle ranging from 0° to 90° were sorted into different phases and DRR of corresponded CT was computed as the input to the network. For this experiment, the relative error between synthetic x-ray projection and realistic ones ranged from 0.04 to 0.11 with an average of 0.087, while the relative error between raw DRR and real x-ray projection ranged from 0.09 to 0.32 with an average of 0.163.
Conclusion: We proposed a Cycle-GAN based DRR rendering method, which could greatly reduce the intensity difference between DRR and x-ray projection using unpaired training data.
Funding Support, Disclosures, and Conflict of Interest: This work was supported by the National Key R&D Program of China under Grant No.2018YFA0704100 and 2018YFA0704101, the National Natural Science Foundation of China under Grant No. 61601012
ePosters
Keywords
X Rays, Image Processing, Digital Imaging
Taxonomy
Contact Email