Sharif Digital Repository

Using Mutual Information (MI) minimization is very common in Blind Source Separation (BSS). However, it is known that gradient descent approaches may trap in local minima of MI in constrained models. In this paper, it is proposed that this problem may be solved using a 'poor' estimation of the derivative of MI  

In this paper, we introduce a novel and robust approach to quantized matrix completion. First, we propose a rank minimization problem with constraints induced by quantization bounds. Next, we form an unconstrained optimization problem by regularizing the rank function with Huber loss. Huber loss is leveraged to control the violation from quantization bounds due to two properties: first, it is differentiable; and second, it is less sensitive to outliers than the quadratic loss. A smooth rank approximation is utilized to endorse lower rank on the genuine data matrix. Thus, an unconstrained optimization problem with differentiable objective function is obtained allowing us to advantage from... 

In this paper, we introduce a new dataset for SAD and evaluate certain common methods such as GMM, DNN, and RNN on it. We have collected our dataset in a semi-supervised approach, using subtitled movies, with a labeling accuracy of 95%. This semi-automatic method can help us collect huge amounts of labeled audio data with very high diversity in language, speaker, and channel. We model the problem of SAD as a classification task to two classes of speech and non-speech. When using GMM for this problem, we use two separate mixtures to model speech and non-speech. In the case of neural networks, we use a softmax layer at the end of the network, with two neurons which represent speech and... 

Deep structures consisting of many layers of nonlinearities have a high potential of expressing complex relations if properly initialized. Autoencoders play a complementary role in training a deep structure by initializing each layer in a greedy unsupervised manner. Due to the high capacity presented by autoencoders, these structures need to be regularized. While mathematical regularizers (based on weight decay, sparsity, etc.) and structural ones (by way of, e.g., denoising and dropout) have been well studied in the literature, quite a few papers have addressed the problem of training autoencoder with non-smooth regularization. In this paper, we address the problem of training autoencoder... 

A simple and systematic approach is developed for modeling and adaptive control of an unknown (or uncertain) chaotic system of the form x(n) = f(X) + g(X)u, using only input-output data obtained from the underlying dynamic system. Two different fuzzy identification methods, i.e. least-squares and gradient descent, are used for identifying the unknown functions f (X) and g (X). Based on the fuzzy modeling, an adaptive controller is devised, which works through sliding mode method. The presented procedure is illustrated by using the chaotic system-modified Duffing's equation as an example, on which simulation results demonstrate the effectiveness of the proposed adaptive algorithm. Copyright ©... 

In feed forward neural networks, hidden layer neurons' saturation conditions, which are the cause of flat spots on the error surface, is one of the main disadvantages of any conventional gradient descent learning algorithm. In this paper, we propose a novel complementary scheme for the learning based on a suitable combination of anti saturated hidden neurons learning process and accelerating methods like the momentum term and the parallel tangent technique. In our proposed method, a normalized saturation criterion (NSC) of hidden neurons, which is introduced in this paper, is monitored during learning process. When the NSC is higher than a specified threshold, it means that the algorithm... 

This paper introduces a novel approach for tuning the parameters of the adaptive network-based fuzzy inference system (ANFIS). In the commonly used training methods, the antecedent and consequent parameters of ANFIS are trained by gradient-based algorithms and recursive least square method, respectively. In this study, a new swarm-based meta-heuristic optimization algorithm, so-called "Cat Swarm Optimization", is used in order to train the antecedent part parameters and gradient descent algorithm is applied for training the consequent part parameters. Experimental results for prediction of Mackey-Glass model and identification of two nonlinear dynamic systems reveal that the performance of... 

A method for controlling chaos when the mathematical model of the system is unknown is presented in this paper. The controller is designed by the pole placement algorithm which provides a linear feedback control method. For calculating the feedback gain, a neural network is used for identification of the system from which the Jacobian of the system in its fixed point can be approximated. The weights of the neural network are adjusted online by the gradient descent algorithm in which the difference between the system output and the network output is considered as the error to be decreased. The method is applied on both discrete-time and continuous-time systems. For continuous-time systems,... 

Semi-Supervised Learning (SSL) has become a topic of recent research that effectively addresses the problem of limited labeled data. Many SSL methods have been developed based on the manifold assumption, among them, the Local and Global Consistency (LGC) is a popular method. The problem with most of these algorithms, and in particular with LGC, is the fact that their naive implementations do not scale well to the size of data. Time and memory limitations are the major problems faced in large-scale problems. In this paper, we provide theoretical bounds on gradient descent, and to overcome the aforementioned problems, a new approximate Newton's method is proposed. Moreover, convergence... 

Deep neural networks have presented superlative performance in many machine learning based perception and recognition tasks, where they have even outperformed human precision in some applications. However, it has been found that human perception system is much more robust to adversarial perturbation, as compared to these artificial networks. It has been shown that a deep architecture with a lower Lipschitz constant can generalize better and tolerate higher level of adversarial perturbation. Smooth regularization has been proposed to control the Lipschitz constant of a deep architecture and in this work, we show how a deep convolutional neural network (CNN), based on non-smooth regularization... 

Classification of video events based on frame-level descriptors is a common approach to video recognition. In the meanwhile, proper encoding of the frame-level descriptors is vital to the whole event classification procedure. While there are some pretty efficient video descriptor encoding methods, temporal ordering of the descriptors is often ignored in these encoding algorithms. In this paper, we show that by taking into account the temporal inter-frame dependencies and tracking the chronological order of video sub-events, accuracy of event recognition is further improved. First, the frame-level descriptors are extracted using convolutional neural networks (CNNs) pre-trained on ImageNet,... 

The main problem with airway segmentation methods which significantly influences their accuracy is leakage into the extra-luminal regions due to thinness of the airway wall during the process of segmentation. This phenomenon potentially makes large regions of lungparenchyma to be wrongly identified as airways. A solution to this problem in the previous methods was based on leak detection followed by reducing leakage during the segmentation process. This has been dealt with adjusting the segmentation parameters and performing the re-segmentation process on the pre-segmented area. This makes the algorithm very exhaustive and more dependent on the user interaction. The method presented here is...