Sharif Digital Repository

In the present contribution, a new approach based on mutual information (MI) is proposed for exploring the independence of feasible solutions in two component systems. Investigating how independent are different feasible solutions can be a way to bridge the gap between independent component analysis (ICA) and multivariate curve resolution (MCR) approaches and, to the best of our knowledge, has not been investigated before. For this purpose, different chromatographic and hyperspectral imaging (HSI) datasets were simulated, considering different noise levels and different degrees of overlap for two-component systems. Feasible solutions were then calculated by both grid search (GS) and... 

The smoothness of the graph signals on predefined/constructed graphs appears in many natural applications of processing unstructured (i.e., graph-based) data. In the case of latent sources being smooth graph signals, blind source separation (BSS) quality can be significantly improved by exploiting graph signal smoothness along with the classic measures of statistical independence. In this paper, we propose a BSS method benefiting from the minimization of mutual information as a well-known independence criterion and also graph signal smoothness term of the estimated latent sources, and show that its performance is superior and fairly robust to the state-of-the-art classic and Graph Signal... 

Semantic segmentation is one of the basic, yet essential scene understanding tasks for an autonomous agent. The recent developments in supervised machine learning and neural networks have enjoyed great success in enhancing the performance of the state-of-the-art techniques for this task. However, their superior performance is highly reliant on the availability of a large-scale annotated dataset. In this letter, we propose a novel fully unsupervised semantic segmentation method, the so-called Information Maximization and Adversarial Regularization Segmentation (InMARS). Inspired by human perception which parses a scene into perceptual groups, rather than analyzing each pixel individually, our... 

A controversial issue in artificial intelligence is human emotion recognition. This paper presents a fuzzy parallel cascades (FPC) model for predicting the continuous subjective emotional appraisal of music by time-varying spectral content of electroencephalogram (EEG) signals. The EEG, along with an emotional appraisal of 15 subjects, was recorded during listening to seven musical excerpts. The emotional appraisement was recorded along the valence and arousal emotional axes as a continuous signal. The FPC model was composed of parallel cascades with each cascade containing a fuzzy logic-based system. The FPC model performance was evaluated using linear regression (LR), support vector... 

We propose two nonlinear regression methods, namely, Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors, while putting no assumption on noise distribution. In both methods, two adversarial networks are trained simultaneously where a regression network outputs predictions and a loss network that estimates mutual information (in AdOR) and KL-divergence (in AdOSE). These methods can be formulated as a minimax two-player game; at equilibrium, AdOR finds a deterministic map between inputs... 

Correlations disguised in various forms underlie a host of important phenomena in classical and quantum systems, such as information and energy exchanges. The quantum mutual information and the norm of the correlation matrix are both considered as proper measures of total correlations. We demonstrate that, when applied to the same system, these two measures can actually show significantly different behavior except at least in two limiting cases: when there are no correlations and when there is maximal quantum entanglement. We further quantify the discrepancy by providing analytic formulas for time derivatives of the measures for an interacting bipartite system evolving unitarily. We argue... 

In this paper, a new measure of correlation is introduced. This measure depends on a parameter α, and is defined in terms of vector-valued Lp norms. The measure is within a constant of the exponential of α-Rényi mutual information, and reduces to the trace norm (total variation distance) for α = 1. We provide some properties and applications of this measure of correlation. In particular, we establish a bound on the secrecy exponent of the wiretap channel (under the total variation metric) in terms of the α-Rényi mutual information according to Csiszár's proposal. © 2019 IEEE  

We present our work on improving the numerator graph for discriminative training using the lattice-free maximum mutual information (MMI) criterion. Specifically, we propose a scheme for creating unconstrained numerator graphs by removing time constraints from the baseline numerator graphs. This leads to much smaller graphs and therefore faster preparation of training supervisions. By testing the proposed un-constrained supervisions using factorized time-delay neural network (TDNN) models, we observe 0.5% to 2.6% relative improvement over the state-of-the-art word error rates on various large-vocabulary speech recognition databases. © 2018 IEEE  

In recent years, end-to-end approaches to automatic speech recognition have received considerable attention as they are much faster in terms of preparing resources. However, conventional multistage approaches, which rely on a pipeline of training hidden Markov models (HMM)-GMM models and tree-building steps still give the state-of-the-art results on most databases. In this study, we investigate flat-start one-stage training of neural networks using lattice-free maximum mutual information (LF-MMI) objective function with HMM for large vocabulary continuous speech recognition. We thoroughly look into different issues that arise in such a setup and propose a standalone system, which achieves... 

In this paper, we introduce a document clustering method based on Sparse Topical Coding, called Cluster-based Sparse Topical Coding. Topic modeling is capable of improving textual document clustering by describing documents via bag-of-words models and projecting them into a topic space. The latent semantic descriptions derived by the topic model can be utilized as features in a clustering process. In our proposed method, document clustering and topic modeling are integrated in a unified framework in order to achieve the highest performance. This framework includes Sparse Topical Coding, which is responsible for topic mining, and K-means that discovers the latent clusters in documents... 

We present our work on end-to-end training of acoustic models using the lattice-free maximum mutual information (LF-MMI) objective function in the context of hidden Markov models. By end-to-end training, we mean flat-start training of a single DNN in one stage without using any previously trained models, forced alignments, or building state-tying decision trees. We use full biphones to enable context-dependent modeling without trees, and show that our end-to-end LF-MMI approach can achieve comparable results to regular LF-MMI on well-known large vocabulary tasks. We also compare with other end-to-end methods such as CTC in character-based and lexicon-free settings and show 5 to 25 percent... 

Similarity measure is a main core of image registration algorithms. Spatially varying intensity distortion is an important challenge, which affects the performance of similarity measures. Correlation among the pixels is the main characteristic of this distortion. Similarity measures such as sum-of-squared-differences (SSD) and mutual information ignore this correlation; hence, perfect registration cannot be achieved in the presence of this distortion. In this paper, we model this correlation with the aid of the low rank matrix theory. Based on this model, we compensate this distortion analytically and introduce rank-regularized SSD (RRSSD). This new similarity measure is a modified SSD based... 

A measure of correlation is said to have the tensorization property if it does not change when computed for i.i.d. copies. More precisely, a measure of correlation between two random variables X, Y denoted by rho (X, Y), has the tensorization property if ρ(Xn, Yn)=ρ (X, Y) where (Xn, Yn) denotes n i.i.d. copies of (X, Y). Two well-known examples of such measures are the maximal correlation and the hypercontractivity ribbon (HC ribbon). We show that the maximal correlation and the HC ribbon are special cases of the new notion of Φ-ribbons, defined in this paper for a class of convex functions Φ. Φ-ribbon reduces to the HC ribbon and the maximal correlation for special choices of Φ, and is a... 

Theorem 5 of A. Gohari, V. Anantharam, IEEE Transactions on Information Theory, vol. 56, no. 8, pp. 3973-3996, 2010, states an upper bound on the secrecy capacity for the source model problem. It has a three page proof given in Appendix B of the paper. Unfortunately, we show that this bound does not provide any improvement over the simpler bound given in Corollary 1 of the paper. We also provide an example of a family of two agent source model problems where the one-way secrecy rate in each direction is zero, but the secrecy rate is nonzero and can be determined exactly as a conditional mutual information. © 1963-2012 IEEE  

Precise acoustical leak detection calls for robust time-delay estimates, which minimize the probability of false alarms in the face of dispersive propagation, multiple reflections, and uncorrelated background noise. Providing evidence that higher order modes and multi-reflected signals behave like sets of correlated noise, this work uses a regression model to optimize residual complexity in the presence of both correlated and uncorrelated noise. This optimized residual complexity (ORC) is highly robust since it takes into account both the level and complexity of noise. The lower complexity of the dispersive modes and multiple reflections, compared to the complexity of the plane mode, points... 

In this paper, we consider a zero error coordination problem wherein the nodes of a network exchange messages to be able to perfectly coordinate their actions with the individual observations of each other. While previous works on coordination commonly assume an asymptotically vanishing error, we assume exact, zero error coordination. Furthermore, unlike previous works that employ the empirical or strong notions of coordination, we define and use a notion of set coordination. This notion of coordination bears similarities with the empirical notion of coordination. We observe that set coordination, in its special case of two nodes with a one-way communication link is equivalent with the Hide... 

The Electrocardiogram (ECG) is widely used for diagnosis of heart diseases. Therefore, the quality of information extracted from the ECG has a vital role. In real recordings, ECG is corrupted by artifacts such as prolonged repolarization, respiration, changes of electrode position, muscle contraction, and power line interface. In this paper, a denoising technique for ECG signals based on Empirical Mode Decomposition (EMD) is proposed. We use Ensemble Empirical Mode Decomposition (EEMD) to overcome the limitations of EMD. Moreover, to overcome the limitations of thresholding methods we use the combination of mutual information and two EMD based interval thresholding approaches. Our new method... 

This paper provides a universal framework for establishing one-shot achievability results for coordination and secrecy problems. The framework is built on our previous framework [Yassaee et al. 13] for proving one-shot achievability results in the context of source and channel coding problems. In the coordination and secrecy problems, one needs to compare an induced distribution by encoding/decoding with an ideal distribution (satisfying some desirable properties) using a suitable criterion. In this paper, we use fidelity as a criterion for measuring the closeness of induced distribution with the ideal distribution. The framework exploits the stochastic mutual information coders at the... 

A function is said to be additive if, similar to mutual information, expands by a factor of n, when evaluated on n i.i.d. repetitions of a source or channel. On the other hand, a function is said to satisfy the tensorization property if it remains unchanged when evaluated on i.i.d. repetitions. Additive rate regions are of fundamental importance in network information theory, serving as capacity regions or upper bounds thereof. Tensorizing measures of correlation have also found applications in distributed source and channel coding problems as well as the distribution simulation problem. Prior to our work only two measures of correlation, namely the hypercontractivity ribbon and maximal... 

Image registration is an important task in medical image processing. Assuming spatially stationary intensity relation among images, conventional area based algorithms such as CC (Correlation Coefficients) and MI (Mutual Information), show weaker results alongside spatially varying intensity distortion. In this research, a structural representation of images is introduced. It allows us to use simpler similarity metrics in multimodal images which are also robust against the mentioned distortion field. The efficiency of this presentation in non-rigid image registration in the presence of spatial varying distortion field is examined. Experimental results on synthetic and real-world data sets...