Loading...

EEG-based Personalized Interpretable Visual Attention Prediction

Behnamnia, Armin | 2022

132 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 55767 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Engineering
  6. Advisor(s): Rabiee, Hamid Reza
  7. Abstract:
  8. Human visual attention is a mapping that determines to what regions of an image human’s eyes focus more while perceiving it. Personalized visual attention is visual attention computed for a specific individual. The importance of visual attention lies in its wide range of applications in computer vision and cognitive science, such as neural encoding, image captioning, self-driving cars, video anomaly detection, image classification, and visual design. One of important aspects of visual attention is personalization, the ability to assign every individual their own, specialized attention map. In this project we aim to utilize EEG signals measured from people’s brain to predict their personalized attention map. EEG brain signals are used to predict the emotion of the person. We improve attention model by detecting objects in the image and predicting their corresponding emotion. Comparing the emotion of the objects inside the image and the emotion of the person, estimated by their measured EEG signals, we emphasize areas most compatible with person’s emotions.We could achieve KL of 0.31 and CC of 0.81 for the attention prediction task on SALICON dataset, an accuracy of 79% on Instagram dataset for image sentiment prediction task, and an accuracy of 94% on DEAP dataset for EEG emotion prediction task.
  9. Keywords:
  10. Interpretability ; Deep Learning ; Neural Networks ; Visual Attention ; Electroencephalogram (EEG)Emotion Recognition ; Electroencephalogram (EEG)Representation Learning ; Personalized Attention

 Digital Object List

 Bookmark

...see more