Loading...

Dynamic iranian sign language recognition using an optimized deep neural network: An implementation via a robotic-based architecture

Basiri, S ; Sharif University of Technology | 2021

401 Viewed
  1. Type of Document: Article
  2. DOI: 10.1007/s12369-021-00819-0
  3. Publisher: Springer Science and Business Media B.V , 2021
  4. Abstract:
  5. Sign language is a non-verbal communication tool used by the deaf. A robust sign language recognition framework is needed to develop Human–Robot Interaction (HRI) platforms that are able to interact with humans via sign language. Iranian sign language (ISL) is composed of both static postures and dynamic gestures of the hand and fingers. In this paper, we present a robust framework using a Deep Neural Network (DNN) to recognize dynamic ISL gestures captured by motion capture gloves in Real-Time. To this end, first, a dataset of fifteen ISL classes was collected in time series; then, this dataset was virtually augmented and pre-processed using the “state-image” method to produce a unique collection of images, each image corresponding to a specific set of sequential data representing a class. Next, by implementing a continuous Genetic algorithm, an optimal deep neural network with the minimum number of weights (trainable parameters) and the maximum overall accuracy was found. Finally, the dataset was fed to the DNN to train the model. The results showed that the optimization process was successful at finding a DNN structure highly suitable for this application, with 99.7% accuracy on the verification (test) data. Then, after implementing the module in a robotic architecture, an HRI experiment was conducted to assess the system’s performance in real-time applications. Preliminary statistical analysis on the standard UTAUT model for eight participants showed that the system can recognize ISL signs quickly and accurately during human–robot interaction. The proposed methodology can be used for other sign languages as no specific characteristics of ISL were used in the preprocessing or training stage. © 2021, The Author(s), under exclusive licence to Springer Nature B.V
  6. Keywords:
  7. Agricultural robots ; Deep neural networks ; Genetic algorithms ; Memory architecture ; Network architecture ; Robotics ; Social robots ; Continuous genetic algorithms ; Motion capture gloves ; Non-verbal communications ; Overall accuracies ; Real-time application ; Robot interactions ; Robotic architectures ; Sign Language recognition ; Neural networks
  8. Source: International Journal of Social Robotics ; 2021 ; 18754791 (ISSN)
  9. URL: https://link.springer.com/article/10.1007/s12369-021-00819-0