Contrast-Enhanced Liver MR Synthesis Using Multi-Modal Sparse Attention Fusion Network
C Jiao1*, Z Fan1, D Ling1, Z Zhu2, W Yang1, (1) University of Southern California, Los Angeles, CA, (2) Xidian University, Xi'an, China
Presentations
PO-GePV-T-12 (Sunday, 7/10/2022) [Eastern Time (GMT-4)]
ePoster Forums
Purpose: MR-guided LINAC has been recently introduced in the clinic, providing versatile sequences to image soft tissues for precision radiotherapy (RT). However, liver tumors or treatment responses are better imaged with contrast agents. The multi-fractionation nature of RT prevents the contrast agent from being administered frequently. We present a multi-modal sparse hybrid attention fusion (MSHAF) network to synthesize T1 contrast-enhanced (T1ce) MR from pre-contrast multi-modal MR images.
Methods: With IRB approval, 165 MR studies (115 training, 20 validation, and 30 testing) were retrospectively solicited from our institutional database. Each study included T2, T1pre, and T1ce. The MR synthesis framework consists of an individual modality feature learning network, a sparse attention fusion with a hybrid operation network, and a pix2pix synthesis network. The individual modality feature learning network learns the modal-specific information from each input. The sparse attention fusion with a hybrid operation network retains the coherent and mutual-complementary features across MR modalities. The pixel-to-pixel synthesis network contains a U-net-based generator synthesizing the contrast-enhanced images and a CNN-based discriminator to differentiate the synthetic image from the ground truth. The MSHAF network was compared with the single modal pix2pix network. The evaluation metrics include peak signal-to-noise ratio (PSNR), structural-similarity-index (SSIM), and mean-squared-error (MSE). Paired student's t-test was performed for comparing models.
Results: On the 30 testing liver MRs, the MSHAF network achieved a PSNR of 26.92, an SSIM of 0.83, and an MSE of 154.60. As a comparison, the pix2pix network with T2 as the single input achieved a PSNR of 26.43, an SSIM of 0.78, and an MSE of 178.45. The MSHAF network showed statistically significant improvements in all metrics tested with p value < 0.05.
Conclusion: A novel multi-modal contrast-enhanced liver MR image synthesis network was successfully constructed to produce synthetic post contrast liver MR images for potential MR-LINAC application.
Keywords
Not Applicable / None Entered.
Taxonomy
IM/TH- MRI in Radiation Therapy: Development (new technology and techniques)
Contact Email