Loading...

Reconstruction of Visual Experience from Brain’s Visual Cortex Data Using
Deep Learning

| 2019

528 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 52339 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Karbalaee Aghajan, Hamid; Soleymani, Mahdieh
  7. Abstract:
  8. e study of the brain’s neural activity is an active research area in computational neuroscience aiming to provide insights about the functionality of the brain as well as dysfunctions that underlie disorders. Functional Magnetic Resonance Imaging (fMRI) plays an important role in brain studies by providing non-invasive records of neural activities during a specific task with location sensitivity. Recent advances in statistics and machine learning offer powerful tools for paern recognition and processing of fMRI data. In this thesis, we decode information recorded via fMRI from the visual cortex to reconstruct images presented to subjects. Current reconstruction methods face numerous challenges such as the need for a preprocessing step of ROI extraction or the limitation to build and test models on data from the same subject. In this research, we utilize the benefits offered by deep learning to propose new approaches for building a generalized model which avoids such limitations. New loss functions are designed resulting in a higher quality for the reconstructed images while circumventing the inherent limitations of fMRI data such as the lack of pixel-level information. In order to avoid common model failures in different test situations like the leave-one-subject-out condition, a collection of domain generalization and adaptation strategies are introduced, rendering a more practical “mind reading” model. We propose a 3D-CNN model for feature extraction from fMRI data that can discover spatial paerns in data and reduces the requirement of ROI selection. e model also provides robustness to the movement noise and brain mismatch across subjects. Our results demonstrate the feasibility of image reconstruction in a leave-one-ubject-out manner as a direction for future fMRI-based brain studies
  9. Keywords:
  10. Mind Reading ; Image Reconstruction ; Deep Learning ; Domain Generalization ; Generative Networks ; Computational Neuro Science ; Brain-Computer Interface (BCI)

 Digital Object List

 Bookmark

No TOC