Loading...

Investigation of a Computer Game Based on Electroencephalogram and Eye Tracker Signals

Nemati, Mohammad | 2023

63 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 56325 (08)
  4. University: Sharif University of Technology
  5. Department: Mechanical Engineering
  6. Advisor(s): Taheri, Alireza; Ghazizadeh, Ali
  7. Abstract:
  8. Video games, as a form of entertainment, have gained widespread attention and usage among all age groups, especially children and adolescents. With a wide variety of game genres and difficulty levels, they offer the opportunity to assess cognitive performance in individuals based on inter-individual differences and variable characteristics such as age, gender, and literacy level. The aim of this research is to study the brain response and gaze dynamics of individuals in a computer game (endless runner) based on electroencephalogram (EEG) signals and eye tracker data. The research process consists of two phases: "Brain Signal Processing in Motor Imagery Tasks" and "Reward and Punishment Processing." In the first phase, an EEG-based brain-computer interface was designed and developed to process the left and right motor imagery signals through brain data and control a running avatar as game input. A dataset of motor imagery available on the internet was processed offline, and an accuracy of 88.91% was achieved. Due to the importance of feature selection, an algorithm based on the Particle Swarm Optimization (PSO) method was designed and tested using a relatively large dataset in combination with the K-means clustering method and inspired by the Genetic Algorithm optimization. This improved the performance by approximately 6% compared to the classical PSO algorithm. Finally, a structure for integrating the Unity and MATLAB software was introduced and tested to implement the brain-computer interface. In the second phase, we focused on a more detailed analysis of brain signals and gaze dynamics during the performance of a video game. Unlike the first phase, we shifted from motor imagery processing to reward and punishment processing. Considering the research needs, the Unity company's “Endless Runner” game was selected and modified for this phase. In this part of the study, data were collected from 10 participants, and we explored the neurocognitive aspects of the data through brain signal processing and gaze modeling. Brain signal processing was conducted in both temporal and spatial domains, and the brain response during collisions with embedded rewards and punishments in the game was examined. It was observed that the brain's P3 component showed a significant difference in amplitude when anticipating rewards during collisions with obstacles compared to collisions with rewards and punishments. This finding indicates a direct relationship between cognitive load of rewards and punishments in the game and the amplitude of the P3 peak in the brain's response to them. In the gaze modeling section, we also observed a significant correlation between participants' gaze coordinates and interactive elements present in the game at each moment. This means that with limited information about interactive elements in the game, it is possible to predict the precise horizontal and vertical gaze coordinates of participants in the next frame (using deep neural networks, specifically transformers) with an average accuracy of 95.93% and an error of less than 50 pixels
  9. Keywords:
  10. Electroencphalogram Signal ; Brain-Computer Interface (BCI) ; Motor Imagery (MI) ; Video Games ; Reward and Punishment Processing

 Digital Object List

 Bookmark

No TOC