Loading...

Combining augmented reality and speech technologies to help deaf and hard of hearing people

Mirzaei, M. R ; Sharif University of Technology | 2012

589 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/SVR.2012.10
  3. Publisher: 2012
  4. Abstract:
  5. Augmented Reality (AR), Automatic Speech Recognition (ASR) and Text-to-Speech Synthesis (TTS) can be used to help people with disabilities. In this paper, we combine these technologies to make a new system for helping deaf people. This system can take the narrator's speech and convert it into a readable text and show it directly on AR display. To improve the accuracy of the system, we use Audio-Visual Speech Recognition (AVSR) as a backup for the ASR engine in noisy environments. In addition, we use the TTS system to make our system more usable for deaf people. The results of testing the system show that its accuracy is over 85 percent on average in different places. Also, the result of a survey shows that more than 90 percent of deaf people on average are very interested in using our system as an assistant in portable devices for communication
  6. Keywords:
  7. Text-to-Speech Synthesis ; AR display ; Audio visual speech recognition ; Automatic speech recognition ; Deaf People ; Noisy environment ; People with disabilities ; Portable device ; Speech technology ; TTS systems ; Audition ; Augmented reality ; Speech recognition ; Speech synthesis ; Handicapped persons
  8. Source: Proceedings - 2012 14th Symposium on Virtual and Augmented Reality, SVR 2012 ; 2012 , Pages 174-181 ; 9780769547251 (ISBN)
  9. URL: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6297574