Loading...

Implementing Adaptive Iranian Sign Language Teaching on RASA Robot

Basiri, Salar | 2020

1162 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 53212 (08)
  4. University: Sharif University of Technology
  5. Department: Mechanical Engineering
  6. Advisor(s): Meghdari, Ali; Taheri, Alireza; Alemi, Minoo
  7. Abstract:
  8. Using social robots as Iranian sign language teaching assistants can be an important step in expanding communication with the deaf in the future. In the literature, it has been shown that user interfaces with adaptive behavior lead to more technological acceptance by the user and increase educational productivity compared to non-adaptive ones. This project aims to empower the RASA robot to perform adaptive Iranian sign language teaching to users, meaning that four of the robot’s outputs will be adaptive to the user: the word to teach, the performance speed, number of repeats, and the robot’s emotional reaction. The first important innovation of this research is the use of a Deep Neural Network (DNN) and the "state-image" method for preprocessing data to use in the sign language recognition module. Another important innovation is the adaptation of the robot’s teaching program to each person as well as the general teaching logic at the same time. In the first phase, Iranian sign language words were collected using a data collection glove, in which each dynamic gesture was converted into a single image by the "state-image" method. Then, using a genetic algorithm, an optimal structure for the deep neural network was obtained, to which the images were fed, and thus the neural network was able to predict gesture patterns. In the second phase, in the ROS environment using Python language, an adaptive teaching architecture was implemented on the RASA robot, which uses fuzzy logic to make the teaching process unique to each person and each session based on users’ performance background. Finally, in the third phase, the teaching performance of the robot was assessed through the statistical analysis of the T-test and Cohen’s d size-effect on the data collected during the teaching sessions and the standard UTAUT questionnaire. The results of the sign recognition system showed that the adopted mechanism can correctly recognize 15 sign language signs with 99.7% accuracy. Statistical results also showed that the adaptation of robot’s teaching was not only well felt by users but also had a significant effect on improving user performance and their attitudes toward human-robot interaction in four items of the standard questionnaire
  9. Keywords:
  10. Deep Neural Networks ; Pattern Recognition ; Social Robotics ; Fuzzy Logic ; Genetic Algorithm ; Persian Sign Language ; Social Robotics

 Digital Object List

 Bookmark

...see more