Loading...

Multimodal Image Registration using Reinforcement Learning-based Methods

Sabour, Amir Hossein | 2023

32 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 56344 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Fatemizadeh, Emadeddin
  7. Abstract:
  8. Image registration is the process of estimating and applying a spatial transformation to a moving image with the aim of spatially aligning it with a fixed image. This allows for the combination of images with complementary information, such as images with different modalities, acquisition times, and even coming from separate individuals, with the purpose of producing more information-rich results. Image registration is a crucial step in many medical applications, such as analyzing the growth and changes of tissue and tumors, preoperative planning, image-guided surgery, radiation therapy planning and various segmentation tasks. Reinforcement learning is a science and mathematical paradigm for generating sequential decisions based on observations and prior experiences through trial and error. In recent years, this field has been exploited as a strong tool in a variety of applications, including image registration, with promising results. In this thesis, several methods and techniques have been proposed for the successful training of agents based on reinforcement learning to register 3D MRI/CT brain images. First off, the fundamental challenges of accomplishing this task have been summarized, followed by the proposed innovations to resolve them. To evaluate the suggested approach, two neural networks with different architectures dubbed “Two Stream” and “Parallel” were trained. A dataset of 37 pairings of brain MRI and CT images were used to train and evaluate the networks. Our simulations show that both networks exhibit favorable evaluation criteria. The average value for the SSIM criterion in the “Two Stream” network is 0.946, 0.976 for the NCC criterion, 0.0014 for the NSSD criterion, and lastly 1.67 for the TRE criterion. Furthermore, the average value for the SSIM criterion obtained from the “Parallel” network is 0.947, 0.977 for the NCC criterion, 0.0014 for the NSSD criterion, and 1.63 for the TRE criterion. Breaking points and intervals of correct performance for each of the trained networks have also been analyzed. These intervals are often substantially greater than the intervals used during training, demonstrating the generalization capabilities of reinforcement learning-based models. According to our simulation results, the methodologies suggested in this thesis have been shown to be effective in the successful training of reinforcement learning agents for MRI/CT registration of 3D brain images
  9. Keywords:
  10. Image Processing ; Medical Images ; Artificial Intelligence ; Reinforcement Learning ; Deep Reinforcement Learning ; Multimodal Images ; Three Dimensional Imaging

 Digital Object List

 Bookmark

No TOC