Loading...

Multimodal Medical Image Registration using Deep Learning

Azampour, Mohammad Farid | 2024

0 Viewed
  1. Type of Document: Ph.D. Dissertation
  2. Language: Farsi
  3. Document No: 57484 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Fatemizadeh, Emadeddin
  7. Abstract:
  8. Medical imaging is a crucial part of modern healthcare, providing vital information for diagnosis, treatment, and monitoring of diseases. Physicians utilize various imaging modalities to obtain valuable information that aids in improving the treatment process. Ultrasound, due to its easy accessibility and lack of side effects, is one of the most commonly used modalities. Its alignment with 3D images such as CT scans and MRI provides useful information for treatment planning. Deep learning in image processing, including image alignment, has made significant advancements. One of the main challenges in this field is the lack of data for training deep networks, which becomes even more crucial in image alignment due to the need for both reference and floating images. Obtaining labeled ultrasound images and paired ultrasound and diagnostic modality images, such as CT scans, is costly and challenging. The aim of this dissertation is to present methods that train deep networks for aligning preoperative images with ultrasound images with minimal need for labeled or paired data. The proposed methods in this dissertation address the issue of alignment in three different scenarios. In the first part, when determining the position of a 2D ultrasound image within a 3D CT scan. In this case, we proposed a simulation-based approach using generative networks and ultrasound physics, achieving an accuracy of 9 mm in translation and 8 degrees in rotation. In the second part, a method was proposed for aligning 3D ultrasound and MRI images. In this method, a physics-based ultrasound generator was used to create pseudo-ultrasound images from MRI images. Additionally, masks from MRI images were used to improve the generated images and alignment results. The final result shows that an accuracy below 4 mm in TRE can be achieved in prostate image alignment using this method. In the third part, the discussion extends the use of GAN to incorporate biomechanical constraints and prior anatomical knowledge to improve ultrasound image alignment. This section focuses on spinal injections, particularly in the lumbar region for treating chronic pain, using ultrasound guidance to provide visual feedback during these procedures. Interpreting spinal ultrasound data is challenging due to noise and shadows caused by the curvature of the spine and bone structures. This section suggests using 3D-to-3D multimodal alignment to align preoperative CT labels of the spine with intraoperative 3D ultrasound, which aggregates 2D B-mode images collected using a tracked transducer. By integrating prior anatomical knowledge into the learning process, the models generate more realistic data during network training and estimate deformation fields that align with the human body's structural features. This improvement leads to better alignment accuracy and prediction of patient position differences between imaging sessions. Overall, this dissertation contributes to the field of medical imaging by developing innovative deep learning-based methods for ultrasound image processing and alignment. The integration of GANs for generating synthetic data and the use of biomechanical constraints and prior anatomical knowledge provide robust solutions for overcoming challenges posed by the lack of annotated data and improving the accuracy of multimodal image alignment. These advancements promise to enhance diagnostic capabilities and clinical applications of ultrasound imaging, ultimately improving patient outcomes in various medical procedures
  9. Keywords:
  10. Multimodal Images ; Similarity Measure ; Deep Learning ; Deep Convolutional Neural Networks ; Medical Ultrasound Images ; Image Registration

 Digital Object List

 Bookmark

...see more