Sharif Digital Repository

Regarding the ever-increasing usage of text generation and analysis in Natural Language Processing field, Language Modeling and Masked Language Modeling have been recently one of the most frequent tasks. Besides, many pretrained models such as BERT have been proposed due to the lack of rich datasets and computational resources among researchers. These models can be finetuned on other datasets in downstream tasks. Although these Transformer-based deep neural networks have performed perfectly in many problems, they still have some shortcomings in a few tasks.Furthermore, structured data like graphs have been recently used extensively in Natural Language Processing and researchers have taken... 

Despite the remarkable success of deep learning algorithms in recent years, it still has a long way to reach the status of human natural intelligence and to acquire the expected self-autonomy. As a result, many researchers in this field have focused on the development of these algorithms while taking inspiration from human cognitive behaviors. One of the disadvantages of current algorithms is the lack of their ability to learn in a continual manner while deployed in the environment. More precisely, deep learning models are not able to gradually gather knowledge from the environment and if they are in a situation of limited access to data, they will suffer from catastrophic forgetting; a... 

Recent advances in deep neural networks have shown significant potential; however, they still face challenges when it comes to non-stationary environments. Continual learning is related to deep neural networks with limited capacity that should perform well on a sequence of tasks. On the other hand, studies have shown that neural networks are sensitive to covariate shifts. But in many cases, the distribution of data varies with time. Domain Adaptation tries to improve the performance of a model on an unlabeled target domain by using the knowledge of other related labeled data coming from a different distribution. Many studies on domain adaptation have optimistic assumptions that are not... 

Isfahan Zirconium Production Plant (ZPP) is one of the most important plants in the nuclear industry of Islamic Republic of Iran. The wastewater of this plant contains high values of Sodium, Chloride, Nitrate and low values of Calcium, Magnesium, Zinc and Zirconium ions. Its release has been banned by international and national law because of the dangers and toxicity of its ions for the environment and living organisms. Therefore, its refinement as a compensation of Iran’s water resources in agriculture and drinking water has received a major consideration. Among the common methods to removal of these ions, membrane process method is selected due to it's major benefits such as not producing... 

Reflective telescopes are devices which commonly use two mirrors, to gather and reflect light rays. In these types of telescopes, the main responsibility of the secondary mirror is to lead the reflected light from primary to detection place. The purpose of this project is to adjust the secondary mirror’s position to reduce the optical errors. So, we are going to study the M2 unit of the telescope. The M2 unit consists of a Hexapod made with six linear actuators in the conventional 3-3 arrangement, and its function is the accurate positioning of the secondary mirror. The main performance required for the M2 unit is the active mirror position adjustment in 5 axes to compensate the existing... 

Increasing the popularity of smartphones and the great ovation of users of mobile apps has turned the app stores to massive software repositories. Therefore, using these repositories can be useful for improving the quality of the program. Since the bridge between users and developers of mobile apps is the comments that users write in the app store, special attention to these comments from developers can make a dramatic improvement in final quality of mobile apps. Hence, in recent years, numerous studies have been conducted around the topic of opinion mining, whose intention was to extract and exert important information from user’s reviews. One of the shortcomings of these studies is the... 

In this thesis, after introducing some preliminary concepts, Stein’s method and Malliavin calculus is discussed. Our approach for introducing Malliavin calculus is based on isonormal Gaussian processes, which is more general and natural than Gaussian noises. After dealing with isonormal Gaussian processes, Wiener chaos and important operators of Malliavin calculus, namely differential, divergence and Ornstein-Uhlenbeck operators are discussed and some relation between them is studied. At last, some connections between Stein’s method and Malliavin calculus is developed. As a result, some exact asymptotics for central limit theorems on Gaussian functionals are obtained. These results are used,... 

Representation learning refers to mapping the input data to another space, usually with lower dimensions than the input space. This task can be helpful in improving the performance of methods in downstream tasks, compression, and improving sample generation in generative models. Representation learning is a problem connected to information theory, and information theory's concepts and quantities are used widely in representation learning models. Besides, the representation learning problem is closely related to latent variable generative models. These models usually learn useful representations in their process of training, implicitly or explicitly. So, the usage of latent variable... 

A large number of machine learning problems are considered as structured output problems in which the goal is to find the mapping function between an input vector to a number of variables in the output side which are statistically correlated. Motivated by the advantages of simultaneous learning of these variables compared to learning them separately, many structured output models have been introduced. Decreasing the sample complexity, increasing the generalization ability and overcoming to noisy data are some of these benefits. So in the first step of this research we concentrate on one of classical but important problems in bioinformatics which is automatic protein function prediction.... 

The existing continual learning methods are mainly focused on fully-supervised scenarios and are still not able to take advantage of unlabeled data available in the environment. Some recent works tried to investigate semi-supervised continual learning (SSCL) settings in which the unlabeled data are available, but it is only from the same distribution as the labeled data. This assumption is still not general enough for real-world applications and restricts the utilization of unsupervised data. In this work, we introduce Open-Set Semi-Supervised Continual Learning (OSSCL), a more realistic semi-supervised continual learning setting in which out-of-distribution (OoD) unlabeled samples in the... 

The goal of drug design is to identify new molecules with a set of desirable properties. The molecular search space is large, discrete, and unstructured, which results in a prolonged construction and testing process of new compounds and requires significant costs. Furthermore, there is a wide variety of appealing options to choose from. Recent advances in the field of machine learning have led to the emergence of generative models that, after training on real examples, can suggest suitable molecules with less time and cost. One of the stages that should be considered in the path of drug production is predicting the properties of the chemical molecule and its effect on the desired protein. By... 

Forecasting and analysing financial markets has always been an interesting research topic for fields ranging from financial sciences to mathematics and statistics. With the rapid development of artificial intelligence in the recent years, there has been a growing interest in using deep neural networks to predict market future trends. The price in these markets is determined by mechanisms of demand and supply. When there is a tendancy to buy a stock, there will be an increase in demand resulting a positive growth for price. On the other hand, when a large group of investors decide to sell their assets, market will experience an increase in supply and subsequently the prices drop. Availability... 

Portfolio optimization is one of the most important issues in capital market and Mathematical Finance. Also in simiulations of financial instruments, in many cases the fluctuations are not independed so we can’t use standard Brownian motion for portfolio optimization and simiulations. In these cases, we should use another kind of Brownian motion which is called fractional Brownian motion. After introducing fractional Brownian motion in chapter 1, we will present its properties in chapter 2 , then at chapter 3 we’ll study stochastic calculus in fractional case and finally in chapter 4 after presenting Stochastic maximum Principle and applying it on a portfolio optimization problem, we will... 

Reinforcement learning is a field of machine learning which is more similar to human training procedures.It uses reward signals to train an agent designed to act in that environment. Deep neural networks enhance the agent’s ability to determine and act better in its complex environment. Most previous works have addressed model-free agents, which ignore modeling details of the environment that in turn can be used to achieve better results. On the other hand, humans utilize a model-based approach in their decision-making process. They use their knowledge to predict the future and choose the action that leads them to a better state. To combine the benefits of model-based and model-free designs,... 

Nowadays, one of the challenges in social robotics is to navigate the robot in social environments with moving elements such as humans. The purpose of this study is to navigate the Arash 2 social robot in a dynamic environment autonomously without encountering moving obstacles (humans). The Arash 2 robot was first simulated in the Gazebo simulator environment in this research. The simultaneous location and mapping (SLAM) technique was implemented on the robot using a lidar sensor to obtain an environment map. Then, using the deep reinforcement learning approach, the neural network developed in the simulation environment was trained and implemented on the robot in the real environment. The... 

Graphs are powerful tools for modeling real-world data. Data analysis using graphs allows us to study the samples relations and identify rich patterns. Although graph modeling can result in a better understanding of data, it requires having strong methods. Graph neural network models have attracted more attention in recent years. These networks are able to simultaneously analyze data features and their relationships to each other and find node representation in low-dimensional feature space. However, due to the novelty of this field, many challenges are still unexplored.In this study, we intend to examine the challenges in this field by focusing on improving the representation of nodes with... 

One of the approaches used in learning theory is using information theoretic tools. The general idea of this approach is that if we show the algorithm did not memorize the dataset, we could guarantee generalization. Noisy channel model is one of the important concepts in this approach. A noisy channel is a lossy process which maps the data to a compressed format.There are two ways to use noisy channel model in literature: input compression and model compression. One of the main results of this thesis is to show that the input compression methods can not explain the generalization of algorithms (despite previous belief). On the contrast by fixing some of the problems faced in the model...