Sharif Digital Repository

Many recent studies show that Augmented Reality (AR) and Automatic Speech Recognition (ASR) technologies can be used to help people with disabilities. Many of these studies have been performed only in their specialized field. Audio-Visual Speech Recognition (AVSR) is one of the advances in ASR technology that combines audio, video, and facial expressions to capture a narrator's voice. In this paper, we combine AR and AVSR technologies to make a new system to help deaf and hard-of-hearing people. Our proposed system can take a narrator's speech instantly and convert it into a readable text and show the text directly on an AR display. Therefore, in this system, deaf people can read the... 

With increasing use of computers in everyday life, improved communication between machines and human is needed. To make a right communication and understand a humankind face which is made in a graphical environment, implementing the audio and visual projects like lip reading, audio and visual speech recognition and lip making are needed. Lack of a complete audio and visual database for this application in Farsi language made us provide a new complete Farsi database for this project that is called SFAVD. It is a unique audio and visual database which in addition to considering Farsi conceptual and speech structure, it considers influence of speech on lip changes. This database is created for... 

Augmented Reality (AR), Automatic Speech Recognition (ASR) and Text-to-Speech Synthesis (TTS) can be used to help people with disabilities. In this paper, we combine these technologies to make a new system for helping deaf people. This system can take the narrator's speech and convert it into a readable text and show it directly on AR display. To improve the accuracy of the system, we use Audio-Visual Speech Recognition (AVSR) as a backup for the ASR engine in noisy environments. In addition, we use the TTS system to make our system more usable for deaf people. The results of testing the system show that its accuracy is over 85 percent on average in different places. Also, the result of a... 

Recently, many researches show Augmented Reality (AR) and Automatic Speech Recognition (ASR) can help people with disabilities. In this paper we implement an innovative system for helping deaf people by combining AR, ASR, and AVSR technologies. This system can instantly take narrator's speech and converts it into readable text and shows it directly on AR display. We show that our system's accuracy becomes over 85 percent on average, by using different ASR engines near using an AVSR engine in different noisy environments. We also show in a survey that more than 90 percent of deaf people on average need such system as assistant in portable devices, near using only text or only sign-language...