Sharif Digital Repository

Meaningful gesture recognition, whose purpose is to interpret human movements, plays a crucial role in various fields such as human and computer interaction, sign language recognition, robot control, and medical applications. Sign language recognition is regarded as the most significant use of gesture recognition by many researchers. Sign languages are the natural medium of communication for millions of deaf people all over the world, and the existence of a sign language recognition system has significantly aided in facilitating communication between deaf individuals and others. Despite numerous studies conducted in this field in recent years, there are still many challenges to continuous... 

In addition to being their primary means of communication with the people around them, sign language for deaf children plays a significant role in shaping their intellectual and cognitive development. Research has shown that the use of robots in education has a substantial impact on the learning process of the learner. As the teaching process requires long-term interaction with different users, the use of adaptable techniques enables the creation of a social robot capable of productive interaction with users of varying performance levels. One of the objectives of this research is to use continual learning algorithms to enhance adaptability in the teaching process. Another goal is to design... 

In Iranian Sign Language (ISL), alongside the movement of fingers, the movement of the lips is also essential for to perform words completely and correctly. The purpose of current study is to provide an automated lip-reading system using deep neural networks and implement it on Rasa social robot; So that the robot can recognize a limited number of specified Persian words. To do this, we propose an automated lip-reading system based on convolutional neural networks and long short-term memories. Convolutional neural networks in extracting features from images and long short-term memories in modeling temporal dynamics have achieved good results. We have also recorded a database in Persian... 

Autism is a neurodevelopmental disorder for which there is no definitive cure and it is increasing every year for people with this disorder. In the last decade, the use of virtual environment to improve the skills of people on the autism spectrum has increased. In this research, a semi-intelligent algorithm was designed for a virtual reality game that could provide a more appropriate step to the user according to the user's responses. Our focus is on designing games to strengthen the child in terms of shared attention and eye contact. In other words, we seek to create a virtual environment with the ability to intelligently adapt to interact with our user and turn it into a suitable coaching... 

Using learning from demonstration approaches is one of the ways to teach a robot how to perform different skills. In classic methods of learning from demonstration, the subject should wear some sensors and devices which is and expensive way. But, with the progresses that have been made in the context of computer vision, it is possible now to achieve to the same results with less cost. Learning from observation is the approach that would be used in this project to do that. By using this method, when robot watches a video, it tries to understand the behavior of the subject and behave in the same way.The purpose of the project is that the robot learns some social (interactive) skills through... 

In this thesis, we introduced a new augmentation method that takes into account the inherent properties of trajectory data and regenerates valid trajectories while preserving all the distinctive features of the main path. Our method uses Dynamic movement primitives (DMP) formulation, which is widely used in path generation in robotics, to manipulate the data in a kinematically accurate way. We implemented the presented method on our Iranian sign language data set by augmenting each group in our data set with a proper form of our DMP data augmentation method. After training our augmented data set with two deep classification models, We achieved 82.95 percent maximum and 77.61 percent mean... 

A social robot is an artificial intelligence-based system designed for humans and other robots. These robots can be used in various settings. The more closely a robot resembles and behaves like a human, the more attention and popularity it will gain. The aim of this research is to humanize the behavior patterns of the robot, especially its gaze in different social situations. The social situations considered in this study are common in human life. In these situations, the robot should make decisions based on the activities that humans are engaged in. The research consists of three parts that design neural networks using LSTM and Transformers architecture. The first part utilizes data... 

In recent years, social robots have used simple and complicated context-aware systems to provide better services for users. The navigation module is one of the most important modules in which a context-aware system can be used; Where a social robot can change the way it navigates with respect to the context that the users are experiencing in a manner that fits the cultural background of the users. This kind of navigation helps the robot to keep its users safe both psychologically and physically. In this research, we present a context-aware navigation method that changes its proxemics due to the group context the users are experiencing. This research consists of three main parts: first, a... 

The hand is an important part of the human body, the loss of all or part of it greatly reduces a person's ability to perform daily tasks. For people who have lost this significant organ, replacing it with an artificial limb that can meet some of their needs is essential. Today, all robotic prostheses use electromyographic (EMG) signals from the remaining muscles of the disabled limb as input signals. Classifying the EMG signal and converting it to a control signal faces serious challenges. Variation of signal properties over time, electrode slippage, muscle fatigue, changes in muscle contraction intensity, and changes in limb position and direction are some of these challenges. Therefore, a... 

Sign language is an essential means of communication for millions of people around the world and serves as their primary language. However, most communication tools and technologies are designed for spoken and written languages, which can create barriers and limitations for the deaf community. By creating a sign language recognition system, we can bridge this communication gap and enable people who use sign language as their primary mode of expression to communicate better with people and their surroundings. This sign language recognition system increases the quality of education, the quality of health services, improves public interactions and creates equal opportunities for the deaf... 

The usage of robots as our friends has been proliferated these days. Knowing that they are going to be used in ordinary houses, we should develop methods and algorithms in order to provide a situation for end-users to program their own robots for their desired tasks. Learning from Demonstrations (LfD) can play a crucial role in this field. In this study, we had taught a non-verbal communication method (pointing) to a robot utilizing LfD. The learning method used was TP-GMM1. The rationale to use this method was that it models all the degrees of freedom together, and we thought it might be an essential parameter to make a movement more natural and understandable which could be two vital... 

Social robots that are designed to interact with people in order to fulfill purposes like education, healthcare, etc. have to behave interactively like human. One of the human’s interactive behaviors is eye gaze. Studying the literature, we found out that in previous researches conducted to control the social robots’ gaze behavior, human gaze behavior was investigated in some limited situations such as two- or three-way conversation in order to extract the pattern of this behavior. Therefore, increasing the variety of studied social situations is a way to fill this gap. In this research we intend to propose an empirical motion-time pattern for human gaze behavior in some different social... 

Video games, as a form of entertainment, have gained widespread attention and usage among all age groups, especially children and adolescents. With a wide variety of game genres and difficulty levels, they offer the opportunity to assess cognitive performance in individuals based on inter-individual differences and variable characteristics such as age, gender, and literacy level. The aim of this research is to study the brain response and gaze dynamics of individuals in a computer game (endless runner) based on electroencephalogram (EEG) signals and eye tracker data. The research process consists of two phases: "Brain Signal Processing in Motor Imagery Tasks" and "Reward and Punishment... 

Using social robots as Iranian sign language teaching assistants can be an important step in expanding communication with the deaf in the future. In the literature, it has been shown that user interfaces with adaptive behavior lead to more technological acceptance by the user and increase educational productivity compared to non-adaptive ones. This project aims to empower the RASA robot to perform adaptive Iranian sign language teaching to users, meaning that four of the robot’s outputs will be adaptive to the user: the word to teach, the performance speed, number of repeats, and the robot’s emotional reaction. The first important innovation of this research is the use of a Deep Neural... 

Social robots are a group of robots designed and built to interact with humans, so the ability to interact emotionally in these robots, especially in interaction with people with hearing and speech disorders, is essential because these people do not have an emotional state sound. Therefore, in this research, an emotional interaction system based on recognizing the user's emotional state from his facial state using deep learning algorithms as well as performing an appropriate emotional response on the robot's face has been designed and implemented on an expressive social robot. In order to design the emotional state recognition system, first, the best model was found on the large AffectNet... 

Nowadays, one of the challenges in social robotics is to navigate the robot in social environments with moving elements such as humans. The purpose of this study is to navigate the Arash 2 social robot in a dynamic environment autonomously without encountering moving obstacles (humans). The Arash 2 robot was first simulated in the Gazebo simulator environment in this research. The simultaneous location and mapping (SLAM) technique was implemented on the robot using a lidar sensor to obtain an environment map. Then, using the deep reinforcement learning approach, the neural network developed in the simulation environment was trained and implemented on the robot in the real environment. The... 

Most of the existing visual SLAM methods heavily rely on a static world assumption and easily fail in dynamic environments. One solution is to eliminate the influence of dynamic objects by introducing deep learning-based semantic information to SLAM systems. In this project, we propose a real-time semantic RGB-D SLAM (built upon RTAB-Map) system for dynamic environments that is capable of detecting moving objects and maintaining a static map for robust camera tracking. Furthermore, we augment the semantic segmentation process using an Extended Kalman filter module to detect temporarily static moving objects by adding centroids to each found dynamic object and calculating their velocity. We... 

Emotions are one of the most important issues which affects daily life and activities. On the other hand, robots play an increasing role in human life and play a fundamental role in meeting our needs. One of these basic roles is empathy and verbal interaction between the robot and human. In this research, participant's emotions were stimulated in two ways: by using OASIS and GAPED image data sets and by instructing the participants to remind about their good or bad memories. During emotional stimulation, EEG signals have been recorded for the training and testing process. The preprocessing of training data includes filtering, removing bad parts of data, removing bad channels and... 

Employing social robots in interactions with children for educational and healthcare objectives could enhance the efficiency of interventions and boost the children’s cognitive and affective outcomes by increasing engagement and providing motivation. This dissertation investigates three main subtopics to determine the efficacy of utilizing social robots in speech therapy sessions in interactions with children suffering from speech and language disorders. We hypothesize that interacting with social robots acting as therapists’ assistants in speech therapy interventions contributes to the formation of language-based communications and improves the individuals’ language skills. The first... 

The process of aging is accompanied by problems that can affect the daily activities of an elderly. Recent studies have shown that the use of virtual reality technology provides a variety of diagnostic methods and clinical systems to prevent and delay cognitive aging to ensure the normal life of the elderly. In this project, efforts have been made to design some virtual reality-based games focusing on memory and recall in the form of three scenarios of finding address, finding car, and virtual-reality based maze game for the elderly to prevent cognitive decline and developing the dementia so that the difficulty of the game level ahead of each elderly can be adjusted according to his or her...