Loading...

Adaptive Teaching of the Iranian Sign Language Based on Continual Learning Algorithms

Memari, Morteza | 2023

42 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 56637 (08)
  4. University: Sharif University of Technology
  5. Department: Mechanical Engineering
  6. Advisor(s): Taheri, Alireza
  7. Abstract:
  8. In addition to being their primary means of communication with the people around them, sign language for deaf children plays a significant role in shaping their intellectual and cognitive development. Research has shown that the use of robots in education has a substantial impact on the learning process of the learner. As the teaching process requires long-term interaction with different users, the use of adaptable techniques enables the creation of a social robot capable of productive interaction with users of varying performance levels. One of the objectives of this research is to use continual learning algorithms to enhance adaptability in the teaching process. Another goal is to design and implement an interactive architecture based on fuzzy logic for adaptive teaching of Iranian sign language to the deaf. This project has been conducted in three phases: teaching sign language vocabulary to the system, designing and implementing adaptive teaching architecture, and clinical studies. In this research, instead of using image data as input, meaningful and relevant features of sign language have been extracted and used as input to the neural network. These features include the coordinates of key points of the hands, coordinates of lip points, and the length and angle of the connecting line between the hands. Based on the extracted features from images, a neural network composed of three transformer encoder modules for input data streams and one transformer encoder module for generating output has been designed. Additionally, the EWC method has been used for continual learning of the neural network. After completing the training process of the neural network, an interactive architecture based on fuzzy logic has been designed for teaching Iranian sign language to users. The designed educational architecture is adaptable at both general and specific levels. To evaluate the performance of the research, a questionnaire based on UTAUT has been designed with minor modifications tailored to the research goals. The neural network introduced in this research has been able to learn a total of 101 Iranian sign language words, including hand and lip movements, in six training tasks and has been able to recognize the complete set of words with an average accuracy of 82.92 percent, and the adaptive teaching architecture interact effectively with different users and adapt itself to them in just four training sessions, so that users, in addition to effective sign language learning, have greater motivation to interact more with the intelligent educational system
  9. Keywords:
  10. Continual Learning ; Fuzzy Logic ; Persian Sign Language ; Human-Machine Interaction ; Adaptive Teaching

 Digital Object List

 Bookmark

No TOC