Image Modality Translation Between MRI and CT Using Diffusion and Score-Matching Algorithms

Notice

This is an unedited manuscript accepted for publication and provided as an Article in Press for early access at the author’s request. The article will undergo copyediting, typesetting, and galley proof review before final publication. Please be aware that errors may be identified during production that could affect the content. All legal disclaimers of the journal apply.

Year : 2025 | Volume : 14 | Issue : 02 | Page : –
    By

    K N Sharma,

  • Tshetiz Dahal,

  • Bimal Nepal,

  1. Assistant Professor, Department of Radio-diagnosis and Imaging, Sikkim Manipal Institute of Medical Sciences Tadong, East Sikkim, India
  2. General Physician, Clinical Researcher, Department of Medicine and Surgery Lugansk State Medical University, Lypnia St. Rivne, Ukraine
  3. Research Scholar, Department of Radiology and Imaging Technology College of Allied and Healthcare Sciences, Teerthankar Mahaveer University Moradabad, Uttar Pradesh, India

Abstract

document.addEventListener(‘DOMContentLoaded’,function(){frmFrontForm.scrollToID(‘frm_container_abs_197392’);});Edit Abstract & Keyword

MRI and CT are the two most widely used medical imaging techniques. Often, doctors need images from both modalities to diagnose conditions accurately and plan treatments like radiation therapy. However, using both MRI and CT can be costly, and it often results in misaligned images. A practical alternative is to use computational methods to convert images from one modality to another—particularly converting MRI images into CT images. In this study, we explore a deep learning approach using diffusion models and score-matching techniques to address this challenge. We specifically employ denoising diffusion probabilistic models along with score-matching techniques, implement four distinct sampling approaches, and evaluate their effectiveness in comparison to conventional models such as generative adversarial networks (GANs) and convolutional neural networks (CNNs). Our results show that diffusion and score-matching models generate synthetic CT images with higher quality than CNN and GAN approaches. We also assess the uncertainty in these models using Monte Carlo simulations and further improve the final image quality by averaging the Monte Carlo outputs. Overall, our research suggests that diffusion and score-matching models not only rival CNNs and GANs in generating cross-modality medical images but also offer a more mathematically grounded and reliable framework.

Keywords: Computed tomography, magnetic resonance imaging, image synthesis, uncertainty estimation, diffusion model, and score-matching model

[This article belongs to Research and Reviews : A Journal of Medical Science and Technology ]

How to cite this article:
K N Sharma, Tshetiz Dahal, Bimal Nepal. Image Modality Translation Between MRI and CT Using Diffusion and Score-Matching Algorithms. Research and Reviews : A Journal of Medical Science and Technology. 2025; 14(02):-.
How to cite this URL:
K N Sharma, Tshetiz Dahal, Bimal Nepal. Image Modality Translation Between MRI and CT Using Diffusion and Score-Matching Algorithms. Research and Reviews : A Journal of Medical Science and Technology. 2025; 14(02):-. Available from: https://journals.stmjournals.com/rrjomst/article=2025/view=0


document.addEventListener(‘DOMContentLoaded’,function(){frmFrontForm.scrollToID(‘frm_container_ref_197392’);});Edit

References

  1. Owrangi AM, Greer PB, Glide-Hurst CK. MRI-only treatment planning: benefits and challenges. Phys Med Biol. 2018 Feb;63(5):05TR01. doi:10.1088/1361-6560/aaaca4.
  2. Kearney V, Ziemer BP, Perry A, Wang T, Chan JW, Ma L, et al. Attention-aware discrimination for MR-to-CT image translation using cycle-consistent generative adversarial networks. Radiol Artif Intell. 2020 Mar;2(2):e190027. doi:10.1148/ryai.2020190027.
  3. Wang G, Kalra M, Murugan V, Xi Y, Gjesteby L, Getzin M, et al. Vision 20/20: Simultaneous CT-MRI—next chapter of multimodality imaging. Med Phys. 2015 Oct;42(10):5879–89. doi:10.1118/1.4929559.
  4. Peng Y, Li M, Grandinetti J, Wang G, Jia X. Top-level design and simulated performance of the first portable CT-MR scanner. arXiv [Preprint]. 2022 [cited 2025 Jul 19]. Available from: https://arxiv.org/abs/2203.15989
  5. Yu B, Wang Y, Wang L, Shen D, Zhou L. Medical image synthesis via deep learning. In: Shen D, Liu T, Yap PT, Huang X, Roth H, editors. Deep learning in medical image analysis: challenges and applications. Cham: Springer; 2020. p. 23–44. doi:10.1007/978-3-030-33128-3_2.
  6. Yang Q, Yan P, Zhang Y, Yu H, Shi Y, Mou X, et al. Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Trans Med Imaging. 2018 Jun;37(6):1348–57. doi:10.1109/TMI.2018.2827462.
  7. Chen H, Zhang Y, Kalra MK, Lin F, Chen Y, Liao P, et al. Low-dose CT with a residual encoder-decoder convolutional neural network. IEEE Trans Med Imaging. 2017 Jun;36(12):2524–35. doi:10.1109/TMI.2017.2715284.
  8. Lyu Q, Shan H, Wang G. MRI super-resolution with ensemble learning and complementary priors. IEEE Trans Comput Imaging. 2020 Jan;6:615–24. doi:10.1109/TCI.2020.2964201.
  9. Hammernik K, Klatzer T, Kobler E, Recht MP, Sodickson DK, Pock T, et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med. 2018 Jun;79(6):3055–71. doi:10.1002/mrm.26977.
  10. Zhang Y, Yu H. Convolutional neural network-based metal artifact reduction in X-ray computed tomography. IEEE Trans Med Imaging. 2018 Jun;37(6):1370–81. doi:10.1109/TMI.2018.2823083.
  11. Gjesteby L, Yang Q, Xi Y, Shan H, Claus B, Jin Y, et al. Deep learning methods for CT image-domain metal artifact reduction. In: Proceedings of SPIE 2017 – Developments in X-ray Tomography XI. Bellingham: SPIE; 2017. vol. 10391, p. 103911G. doi:10.1117/12.2274427.
  12. Nie D, Cao X, Gao Y, Wang L, Shen D. Estimating CT image from MRI data using 3D fully convolutional networks. In: Cardoso MJ, Arbel T, Carneiro G, Syeda-Mahmood T, Martel A, Maier-Hein L, et al., editors. Deep learning and data labeling for medical applications. Cham: Springer; 2016. p. 170–8. (Lecture Notes in Computer Science; vol. 10008). doi:10.1007/978-3-319-46976-8_18.
  13. Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Med Phys. 2017 Feb;44(4):1408–19. doi:10.1002/mp.12155.
  14. Leynes AP, Yang J, Wiesinger F, Kaushik SS, Shanbhag DD, Seo Y, et al. Direct pseudoCT generation for pelvis PET/MRI attenuation correction using deep convolutional neural networks with multiparametric MRI: zero echo-time and Dixon deep pseudoCT (ZeDD-CT). J Nucl Med. 2017 Oct. doi:10.2967/jnumed.117.198051.
  15. Chartsias A, Joyce T, Dharmakumar R, Tsaftaris SA. Adversarial image synthesis for unpaired multi-modal cardiac data. In: Gooya A, Glocker B, Oguz I, editors. Simulation and synthesis in medical imaging. Cham: Springer; 2017. p. 3–13. doi:10.1007/978-3-319-68127-6_1.
  16. Nie D, Trullo R, Lian J, Wang L, Petitjean C, Ruan S, et al. Medical image synthesis with deep convolutional adversarial networks. IEEE Trans Biomed Eng. 2018 Dec;65(12):2720–30. doi:10.1109/TBME.2018.2814538.
  17. Emami H, Dong M, Nejad-Davarani SP, Glide-Hurst CK. Generating synthetic CTs from magnetic resonance images using generative adversarial networks. Med Phys. 2018 Jun;45(8):3627–36. doi:10.1002/mp.13047.
  18. Hiasa Y, Otake Y, Takao M, Matsuoka T, Takashima K, Carass A, et al. Cross-modality image synthesis from unpaired data using CycleGAN. In: Cardoso MJ, Arbel T, Carneiro G, Syeda-Mahmood T, Martel A, Maier-Hein L, et al., editors. Simulation and synthesis in medical imaging. Cham: Springer; 2018. p. 31–41. doi:10.1007/978-3-030-00536-8_4.
  19. Zhang Z, Yang L, Zheng Y. Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2018. p. 9242–51. doi:10.1109/CVPR.2018.00963.
  20. Ben-Cohen A, Klang E, Raskin SP, Amitai MM, Greenspan H. Virtual PET images from CT data using deep convolutional networks: initial results. In: Gooya A, Glocker B, Oguz I, editors. Simulation and synthesis in medical imaging. Cham: Springer; 2017. p. 49–57. doi:10.1007/978-3-319-68127-6_6.
  21. Armanious K, Jiang C, Fischer M, Küstner T, Hepp T, Nikolaou K, et al. MedGAN: Medical image translation using GANs. Comput Med Imaging Graph. 2020 Jan;79:101684. doi:10.1016/j.compmedimag.2019.101684.
  22. Bi L, Kim J, Kumar A, Feng D, Fulham M. Synthesis of positron emission tomography (PET) images via multi-channel generative adversarial networks (GANs). In: Liu B, Rekik I, Adeli E, Shen D, editors. Molecular Imaging, and Analysis of Moving Body Organs, and Stroke Imaging and Treatment. RAMBO, CMMI, SWITCH 2017. Cham: Springer; 2017. p. 43–51. doi:10.1007/978-3-319-67564-0_5.
  23. Choi H, Lee DS. Generation of structural MR images from amyloid PET: application to MR-less quantification. J Nucl Med. 2018 Jul;59(7):1111–7. doi:10.2967/jnumed.117.199414.
  24. Wei W, Poirion E, Bodini B, Durrleman S, Ayache N, Stankoff B, et al. Learning myelin content in multiple sclerosis from multimodal MRI through adversarial training. In: Frangi AF, Schnabel JA, Davatzikos C, Alberola-López C, Fichtinger G, editors. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2018. Cham: Springer; 2018. p. 514–22. doi:10.1007/978-3-030-00931-1_59.
  25. Cai J, Zhang Z, Cui L, Zheng Y, Yang L. Towards cross-modal organ translation and segmentation: A cycle-and shape-consistent generative adversarial network. Med Image Anal. 2019 Feb;52:174–84. doi:10.1016/j.media.2018.12.002.
  26. Li W, Li Y, Qin W, Liang X, Xu J, Xiong J, et al. Magnetic resonance image (MRI) synthesis from brain computed tomography (CT) images based on deep learning methods for magnetic resonance (MR)-guided radiotherapy. Quant Imaging Med Surg. 2020 Jun;10(6):1223–36. doi:10.21037/qims-19-885.
  27. Bahrami A, Karimian A, Fatemizadeh E, Arabi H, Zaidi H. A new deep convolutional neural network design with efficient learning capability: Application to CT image synthesis from MRI. Med Phys. 2020 Jul;47(7):5158–71. doi:10.1002/mp.14418.
  28. Boni KNB, Klein J, Vanquin L, Wagner A, Lacornerie T, Pasquier D, et al. MR to CT synthesis with multicenter data in the pelvic area using a conditional generative adversarial network. Phys Med Biol. 2020 Apr;65(7):075002. doi:10.1088/1361-6560/ab7633.
  29. Ben-Cohen A, Klang E, Raskin SP, Soffer S, Ben-Haim S, Konen E, et al. Cross-modality synthesis from CT to PET using FCN and GAN networks for improved automated lesion detection. Eng Appl Artif Intell. 2019 Feb;78:186–94. doi:10.1016/j.engappai.2018.11.013.
  30. Tao L, Fisher J, Anaya E, Li X, Levin CS. Pseudo CT image synthesis and bone segmentation from MR images using adversarial networks with residual blocks for MR-based attenuation correction of brain PET data. IEEE Trans Radiat Plasma Med Sci. 2020 Mar;5(2):193–201. doi:10.1109/TRPMS.2020.2989073.
  31. Hu S, Lei B, Wang S, Wang Y, Feng Z, Shen Y. Bidirectional mapping generative adversarial networks for brain MR to PET synthesis. IEEE Trans Med Imaging. 2022 Jan;41(1):145–57. doi:10.1109/TMI.2021.3107013.
  32. Zhang J, He X, Qing L, Gao F, Wang B. BPGAN: Brain PET synthesis from MRI using generative adversarial network for multi-modal Alzheimer’s disease diagnosis. Comput Methods Programs Biomed. 2022 Apr;217:106676. doi:10.1016/j.cmpb.2022.106676.
  33. Song Y, Ermon S. Generative modeling by estimating gradients of the data distribution. In: Wallach H, Larochelle H, Beygelzimer A, d’Alché-Buc F, Fox E, Garnett R, editors. Advances in Neural Information Processing Systems. NeurIPS 2019. Vol. 32. Red Hook, NY: Curran Associates Inc.; 2019. p. 11918–30.
  34. Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H, editors. Advances in Neural Information Processing Systems. NeurIPS 2020. Vol. 33. Red Hook, NY: Curran Associates Inc.; 2020. p. 6840–51.
  35. Song Y, Sohl-Dickstein J, Kingma DP, Kumar A, Ermon S, Poole B. Score-based generative modeling through stochastic differential equations. In: International Conference on Learning Representations (ICLR 2021). OpenReview.net; 2021.
  36. Croitoru F-A, Hondru V, Ionescu RT, Shah M. Diffusion models in vision: A survey. arXiv [Preprint]. 2022 [cited 2025 Jul 19]. Available from: https://arxiv.org/abs/2209.04747
  37. Sohl-Dickstein J, Weiss E, Maheswaranathan N, Ganguli S. Deep unsupervised learning using nonequilibrium thermodynamics. In: Bach F, Blei D, editors. Proceedings of the 32nd International Conference on Machine Learning (ICML 2015). Vol. 37. PMLR; 2015. p. 2256–65.
  38. Vincent P. A connection between score matching and denoising autoencoders. Neural Comput. 2011 Jul;23(7):1661–74. doi:10.1162/NECO_a_00142.
  39. Nyholm T, Svensson S, Andersson S, Jonsson J, Sohlin M, Gustafsson C, et al. MR and CT data with multiobserver delineations of organs in the pelvic area—part of the Gold Atlas project. Med Phys. 2018 Jan;45(3):1295–300. doi:10.1002/mp.12748.
  40. Saharia C, Ho J, Chan W, Salimans T, Fleet DJ, Norouzi M. Image super-resolution via iterative refinement. IEEE Trans Pattern Anal Mach Intell. 2022;doi:10.1109/TPAMI.2022.3204461.
  41. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer; 2015. p. 234–41. doi:10.1007/978-3-319-24574-4_28.
  42. Song Y, Shen L, Xing L, Ermon S. Solving inverse problems in medical imaging with score-based generative models. In: International Conference on Learning Representations (ICLR 2022). OpenReview.net; 2022.
  43. Dhariwal P, Nichol AQ. Diffusion models beat GANs on image synthesis. In: Ranzato M, Beygelzimer A, Dauphin Y, Liang PS, Vaughan JW, editors. Advances in Neural Information Processing Systems. NeurIPS 2021. Vol. 34. Curran Associates, Inc.; 2021. p. 8780–94.
  44. Liu X, Park DH, Azadi S, Zhang G, Chopikyan A, Hu Y, et al. More control for free! Image synthesis with semantic diffusion guidance. arXiv [Preprint]. 2021 [cited 2025 Jul 19]. Available from: https://arxiv.org/abs/2112.05744
  45. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A. Improved training of Wasserstein GANs. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, et al., editors. Advances in Neural Information Processing Systems. NeurIPS 2017. Vol. 30. Curran Associates, Inc.; 2017. p. 5769–79.
  46. Johnson J, Alahi A, Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution. In: Leibe B, Matas J, Sebe N, Welling M, editors. European Conference on Computer Vision – ECCV 2016. Cham: Springer; 2016. p. 694–711. doi:10.1007/978-3-319-46475-6_43.
  47. Song J, Meng C, Ermon S. Denoising diffusion implicit models. In: International Conference on Learning Representations (ICLR 2021). OpenReview.net; 2021.
  48. Lu C, Zhou Y, Bao F, Chen J, Li C, Zhu J. DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. arXiv [Preprint]. 2022 [cited 2025 Jul 19]. Available from: https://arxiv.org/abs/2206.00927
  49. Ma H, Zhang L, Zhu X, Feng J. Accelerating score-based generative models with preconditioned diffusion sampling. arXiv [Preprint]. 2022 [cited 2025 Jul 19]. Available from: https://arxiv.org/abs/2207.02196
  50. Lyu Z, Xu X, Yang C, Lin D, Dai B. Accelerating diffusion models via early stop of the diffusion process. arXiv [Preprint]. 2022 [cited 2025 Jul 19]. Available from: https://arxiv.org/abs/2205.12524
  51. Chung H, Sim B, Ye JC. Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022. Piscataway, NJ: IEEE; 2022. p. 12413–22.
Regular Issue Subscription Original Research
Volume 14
Issue 02
Received 10/07/2025
Accepted 20/07/2025
Published 22/07/2025
Publication Time 12 Days
307

[first_name] [last_name]

My IP

PlumX Metrics