Sharif Digital Repository

We explore the dictionary learning problem for sparse representation when the signals are dependent. In this paper, a first-order Markovian model is considered for dependency of the signals, that has many applications especially in medical signals. It is shown that the considered dependency among the signals can degrade the performance of the existing dictionary learning algorithms. Hence, we propose a method using the Maximum Log-likelihood Estimator (MLE) and the Expectation Minimization (EM) algorithm to learn the dictionary from the signals generated under the first-order Markovian model. Simulation results show the efficiency of the proposed method in comparison with the... 

Although exemplar based approaches have shown good accuracy in classification problems, some limitations are observed in the accuracy of exemplar based automatic speech recognition (ASR) applications. The main limitation of these algorithms is their high computational complexity which makes them difficult to extend to ASR applications. In this paper, an N-best class selector is introduced based on sparse representation (SR) and a tree search strategy. In this approach, the classification is fulfilled in three steps. At first, the set of similar training samples for the specific test sample is selected by k-dimensional (KD) tree search algorithm. Then, an SR based N-best class selector is... 

This paper proposes an image watermarking scheme for copyright protection based on Independent Component Analysis (ICA). In the suggested scheme, embedding is carried out in cumulative form in spatial domain and ICA is used for watermark extraction. For extraction there is no need to access the original image or the watermark, and extraction is carried out only with two watermarked images. Experimental results show that the new method has better quality than famous methods [1], [2], [3] in spatial or frequency domain and is robust against various attacks. Noise addition, resizing, low pass filtering, multiple marks, gray-scale reduction, rotation, JPEG compression, and cropping are some... 

In this paper, we address the problem of recovering degraded images using multivariate Gaussian mixture model (GMM) as a prior. The GMM framework in our method for image restoration is based on the assumption that the accumulation of similar patches in a neighborhood are derived from a multivariate Gaussian probability distribution with a specific covariance and mean. Previous methods of image restoration with GMM have not considered spatial (geometric) distance between patches in clustering. Our conducted experiments show that in the case of constraining Gaussian estimates into a finite-sized windows, the patch clusters are more likely to be derived from the estimated multivariate Gaussian... 

In this paper, a property for sparse recovery algorithms, called invariancy, is introduced. The significance of invariancy is that the performance of the algorithms with this property is less affected when the sensing (i.e., the dictionary) is ill-conditioned. This is because for this kind of algorithms, there exists implicitly an equivalent well-conditioned problem, which is being solved. Some examples of sparse recovery algorithms will also be considered and it will be shown that some of them, such as SL0, Basis Pursuit (using interior point LP solver), FOCUSS, and hard thresholding algorithms, are invariant, and some others, like Matching Pursuit and SPGL1, are not. Then, as an... 

In this paper we propose a new algorithm for learning low-dimensional linear subspaces. Our proposed algorithm performs by sequentially finding some low-dimensional subspaces on which a set of training data lies. Each subspace is found in such a way that the number of signals lying on (or near to) it is maximized. Once we found a subset of the training data that is sufficiently close to a subspace, then we omit these signals from the set of training signals and repeat the procedure for the remaining signals until all training signals are assigned to a subspace. This data reduction procedure results in a significant improvement to the runtime of our algorithm. We then propose a robust version... 

Recently, great attention was intended toward overcomplete dictionaries and the sparse representations they can provide. In a wide variety of signal processing problems, sparsity serves a crucial property leading to high performance. Decomposition of a given signal over two or more dictionaries with sparse coefficients is investigated in this paper. This kind of decomposition is useful in many applications such as inpainting, denoising, demosaicing, speech source separation, high-quality zooming and so on. This paper addresses a novel technique of such a decomposition and investigates this idea through inpainting of images which is the process of reconstructing lost or deteriorated parts of... 

Learning sparsifying dictionaries from a set of training signals has been shown to have much better performance than pre-designed dictionaries in many signal processing tasks, including image enhancement. To this aim, numerous practical dictionary learning (DL) algorithms have been proposed over the last decade. This paper introduces an accelerated DL algorithm based on iterative proximal methods. The new algorithm efficiently utilizes the iterative nature of DL process, and uses accelerated schemes for updating dictionary and coefficient matrix. Our numerical experiments on dictionary recovery show that, compared with some well-known DL algorithms, our proposed one has a better convergence... 

A significant challenge in spectrum sensing is to lessen the signal to noise ratio needed to detect the presence of primary users while the noise level may also be unknown. To meet this challenge, multi-antenna based techniques possess a greater efficiency compared to other algorithms. In a typical compact multi-antenna system, due to small interelement spacing, mutual coupling between thermal noises of adjacent receivers is significant. In this paper, unlike most of the spectrum sensing algorithms which assume spatially uncorrelated noise, the noises on the adjacent antennas can have arbitrary correlations. Also, in contrast to some other algorithms, no prior assumption is made on the... 

The theory of compressive sampling involves making random linear projections of a signal. Provided signal is sparse in some basis, small number of such measurements preserves the information in the signal, with high probability. Following the success in signal reconstruction, compressive framework has recently proved useful in classification, particularly hypothesis testing. In this paper, conventional random projection scheme is first extended to the image domain and the key notion of concentration of measure is closely studied. Findings are then employed to develop a 2D compressive classifier (2D-CC) for sparse images. Finally, theoretical results are validated within a realistic... 

Spectral manifold learning techniques have recently found extensive applications in machine vision. The common strategy of spectral algorithms for manifold learning is exploiting the local relationships in a symmetric adjacency graph, which is typically constructed using k-nearest neighborhood (k-NN) criterion. In this paper, with our focus on locally linear embedding as a powerful and well-known spectral technique, shortcomings of k-NN for construction of the adjacency graph are first illustrated, and then a new criterion, namely k/K-nearest neighborhood (k/K-NN) is introduced to overcome these drawbacks. The proposed criterion involves finding the sparsest representation of each sample in... 

The theory of compressive sampling involves making random linear projections of a signal. Provided signal is sparse in some basis, small number of such measurements preserves the information in the signal, with high probability. Following the success in signal reconstruction, compressive framework has recently proved useful in classification. In this paper, conventional random projection scheme is first extended to the image domain and the key notion of concentration of measure is studied. Findings are then employed to develop a 2D compressive classifier (2DCC) for sparse images. Finally, theoretical results are validated within a realistic experimental framework. ©2009 IEEE  

Spectral manifold learning techniques have recently found extensive applications in machine vision. The common strategy of spectral algorithms for manifold learning is exploiting the local relationships in a symmetric adjacency graph, which is typically constructed using k -nearest neighborhood (k-NN) criterion. In this paper, with our focus on locally linear embedding as a powerful and well-known spectral technique, shortcomings of k-NN for construction of the adjacency graph are first illustrated, and then a new criterion, namely k/K-nearest neighborhood (k/K-NN) is introduced to overcome these drawbacks. The proposed criterion involves finding the sparsest representation of each sample in... 

In this paper, a sparse component analysis algorithm is presented for the case in which the number of sources is less than or equal to the number of sensors, but the channel (mixing matrix) is time-varying. The method is based on a smoothed l0 norm for the sparsity criteria, and takes advantage of the idea that sparsity of the sources is decreased when they are mixed. The method is able to separate synthetic and speech data, which require very weak sparsity restrictions. It can separate up to 50 mixed signals while being adaptive to channel variation and robust against noise. © 2007 IEEE  

This paper is a survey of semi-blind source separation approaches. Since Gaussian iid signals are not separable, simplest priors suggest to assume non Gaussian iid signals, or Gaussian non iid signals. Other priors can also been used, for instance discrete or bounded sources, positivity, etc. Although providing a generic framework for semi-blind source separation, Sparse Component Analysis and Bayesian ICA will just sketched in this paper, since two other survey papers develop in depth these approaches. © 2006 i6doc.com publication. All rights reserved  

In this paper, a nonparametric "gradient" of the mutual information is first introduced. It is used for showing that mutual information has no local minima. Using the introduced "gradient", two general gradient based approaches for minimizing mutual information in a parametric model are then presented. These approaches are quite general, and principally they can be used in any mutual information minimization problem. In blind source separation, these approaches provide powerful tools for separating any complicated (yet separable) mixing model. In this paper, they are used to develop algorithms for separating four separable mixing models: linear instantaneous, linear convolutive, post... 

In this paper, we address the matrix completion problem and propose a novel algorithm based on a smoothed rank function (SRF) approximation. Among available algorithms like FPCA and OptSpace, there is no solution that can simultaneously cover wide range of easy and hard problems. This new algorithm provides accurate results in almost all scenarios with a reasonable run time. It especially has low execution time in hard problems where other methods need long time to converge. Furthermore, when the rank is known in advance and is high, our method is very faster than previous methods for the same accuracy. The main idea of the algorithm is based on a continuous and differentiable approximation... 

Suppose that a solution x to an underdetermined linear system b=Ax is given. x is approximately sparse meaning that it has a few large components compared to other small entries. However, the total number of nonzero components of x is large enough to violate any condition for the uniqueness of the sparsest solution. On the other hand, if only the dominant components are considered, then it will satisfy the uniqueness conditions. One intuitively expects that x should not be far from the true sparse solution x0. It was already shown that this intuition is the case by providing upper bounds on ||x-x0|| which are functions of the magnitudes of small components of x but independent from x0. In... 

In this paper an Extended Kalman Filter (EKF) has been proposed for the filtering of noisy ECG signals. The method is based on a modified nonlinear dynamic model, previously introduced for the generation of synthetic ECG signals. An automatic parameter selection method has also been suggested, to adapt the model with a vast variety of normal and abnormal ECG signals. The results show that the EKF output is able to track the original ECG signal shape even in the most noisiest epochs of the ECG signal. The proposed method may serve as an efficient filtering procedure for applications such as the noninvasive extraction of fetal cardiac signals from maternal abdominal signals. © 2005 IEEE