Sharif Digital Repository

This paper provides a universal framework for establishing one-shot achievability results for coordination and secrecy problems. The framework is built on our previous framework [Yassaee et al. 13] for proving one-shot achievability results in the context of source and channel coding problems. In the coordination and secrecy problems, one needs to compare an induced distribution by encoding/decoding with an ideal distribution (satisfying some desirable properties) using a suitable criterion. In this paper, we use fidelity as a criterion for measuring the closeness of induced distribution with the ideal distribution. The framework exploits the stochastic mutual information coders at the... 

A function is said to be additive if, similar to mutual information, expands by a factor of n, when evaluated on n i.i.d. repetitions of a source or channel. On the other hand, a function is said to satisfy the tensorization property if it remains unchanged when evaluated on i.i.d. repetitions. Additive rate regions are of fundamental importance in network information theory, serving as capacity regions or upper bounds thereof. Tensorizing measures of correlation have also found applications in distributed source and channel coding problems as well as the distribution simulation problem. Prior to our work only two measures of correlation, namely the hypercontractivity ribbon and maximal... 

In CHES 2008 a generic side-channel distinguisher, Mutual Information, has been introduced to be independent of the relation between measurements and leakages as well as between leakages and data processed. Assuming a Gaussian model for the side-channel leakages, correlation power analysis (CPA) is capable of revealing the secrets efficiently. The goal of this paper is to compare mutual information analysis (MIA) and CPA when leakage of the target device fits into a Gaussian assumption. We first theoretically examine why MIA can reveal the correct key guess amongst other hypotheses, and then compare it with CPA proofs. As our theoretical comparison confirms and shown recently in ACNS 2009... 

A video activity analysis is presented based on 3D wavelet transform. Marginal and joint statistics as well as mutual information estimates are extracted. Marginal histograms are approximated by Generalized Gaussian Density (GGD) functions. The mutual information between coefficients -as a quantitative estimate of joint statistics- decreases when the activity in the video increases. The relationship between kurtosis graphs, extracted from joint distributions and video activity, is deduced. Results show that the type of activity in the video can be figured out from Kurtosis curves. The GGD and the Kullback-Leibler distance (KLD) are used to retrieve and locate 96% of videos properly  

Correlations disguised in various forms underlie a host of important phenomena in classical and quantum systems, such as information and energy exchanges. The quantum mutual information and the norm of the correlation matrix are both considered as proper measures of total correlations. We demonstrate that, when applied to the same system, these two measures can actually show significantly different behavior except at least in two limiting cases: when there are no correlations and when there is maximal quantum entanglement. We further quantify the discrepancy by providing analytic formulas for time derivatives of the measures for an interacting bipartite system evolving unitarily. We argue... 

We propose an algorithm based on mutual information to address the problem of online reconstruction of dynamic MRI from partial k-space measurements. Most of previous compressed sensing (CS) based methods successfully leverage sparsity constraint for offline reconstruction of MR images, yet they are not used in online applications due to their complexities. In this paper, we formulate the reconstruction as a constraint optimization problem and try to maximize the mutual information between the current and the previous time frames. Conjugate gradient method is used to solve the optimization problem. Using Cartesian mask to undersample k-space measurements, the proposed method reduces... 

In this paper, a nonparametric "gradient" of the mutual information is first introduced. It is used for showing that mutual information has no local minima. Using the introduced "gradient", two general gradient based approaches for minimizing mutual information in a parametric model are then presented. These approaches are quite general, and principally they can be used in any mutual information minimization problem. In blind source separation, these approaches provide powerful tools for separating any complicated (yet separable) mixing model. In this paper, they are used to develop algorithms for separating four separable mixing models: linear instantaneous, linear convolutive, post... 

In the present contribution, a new approach based on mutual information (MI) is proposed for exploring the independence of feasible solutions in two component systems. Investigating how independent are different feasible solutions can be a way to bridge the gap between independent component analysis (ICA) and multivariate curve resolution (MCR) approaches and, to the best of our knowledge, has not been investigated before. For this purpose, different chromatographic and hyperspectral imaging (HSI) datasets were simulated, considering different noise levels and different degrees of overlap for two-component systems. Feasible solutions were then calculated by both grid search (GS) and... 

In this article, we examine Independent Component Analysis (ICA) and the concept of Mutual information (MI) as a quantitative measure of independence from the point of view of analytical chemistry. We compare results obtained by different ICA methods with results obtained by Multivariate Curve Resolution Alternating Least Squares (MCR-ALS). These results have shown that, when non-negativity constraints are applied, values of MI increase considerably and the resolved components cannot anymore be considered to be independent (i.e. they can only be considered to be the " least dependent" components). MI values of profiles resolved by MCR-ALS and ICA did not differ significantly when... 

Similarity measure is an important key in image registration. Most traditional intensity-based similarity measures (e.g. sum-of-squared-differences, correlation coefficient, mutual information and correlation ratio) assume a stationary image and pixel-by-pixel independence. These similarity measures ignore the correlation among pixel intensities; hence, a perfect image registration cannot be achieved especially in the presence of spatially varying intensity distortions and outlier objects that appear in one image but not in the other. It is supposed here that non-stationary intensity distortion (such as bias field) has a sparse representation in the transformation domain. Based on this... 

Video activity analysis is used in various video applications such as human action recognition, video retrieval, video archiving. In this paper, we propose to apply 3D wavelet transform statistics to natural video signals and employ the resulting statistical attributes for video modeling and analysis. From the 3D wavelet transform, we investigate the marginal and joint statistics as well as the Mutual Information (MI) estimates. We show that marginal histograms are approximated quite well by Generalized Gaussian Density (GGD) functions; and the MI between coefficients decreases when the activity level increases in videos. Joint statistics attributes are applied to scene activity grouping,... 

In this paper, we consider a zero error coordination problem wherein the nodes of a network exchange messages to be able to perfectly coordinate their actions with the individual observations of each other. While previous works on coordination commonly assume an asymptotically vanishing error, we assume exact, zero error coordination. Furthermore, unlike previous works that employ the empirical or strong notions of coordination, we define and use a notion of set coordination. This notion of coordination bears similarities with the empirical notion of coordination. We observe that set coordination, in its special case of two nodes with a one-way communication link is equivalent with the Hide... 

The Electrocardiogram (ECG) is widely used for diagnosis of heart diseases. Therefore, the quality of information extracted from the ECG has a vital role. In real recordings, ECG is corrupted by artifacts such as prolonged repolarization, respiration, changes of electrode position, muscle contraction, and power line interface. In this paper, a denoising technique for ECG signals based on Empirical Mode Decomposition (EMD) is proposed. We use Ensemble Empirical Mode Decomposition (EEMD) to overcome the limitations of EMD. Moreover, to overcome the limitations of thresholding methods we use the combination of mutual information and two EMD based interval thresholding approaches. Our new method... 

Image registration is an important task in medical image processing. Assuming spatially stationary intensity relation among images, conventional area based algorithms such as CC (Correlation Coefficients) and MI (Mutual Information), show weaker results alongside spatially varying intensity distortion. In this research, a structural representation of images is introduced. It allows us to use simpler similarity metrics in multimodal images which are also robust against the mentioned distortion field. The efficiency of this presentation in non-rigid image registration in the presence of spatial varying distortion field is examined. Experimental results on synthetic and real-world data sets... 

Precise acoustical leak detection calls for robust time-delay estimates, which minimize the probability of false alarms in the face of dispersive propagation, multiple reflections, and uncorrelated background noise. Providing evidence that higher order modes and multi-reflected signals behave like sets of correlated noise, this work uses a regression model to optimize residual complexity in the presence of both correlated and uncorrelated noise. This optimized residual complexity (ORC) is highly robust since it takes into account both the level and complexity of noise. The lower complexity of the dispersive modes and multiple reflections, compared to the complexity of the plane mode, points... 

In this paper, we introduce a document clustering method based on Sparse Topical Coding, called Cluster-based Sparse Topical Coding. Topic modeling is capable of improving textual document clustering by describing documents via bag-of-words models and projecting them into a topic space. The latent semantic descriptions derived by the topic model can be utilized as features in a clustering process. In our proposed method, document clustering and topic modeling are integrated in a unified framework in order to achieve the highest performance. This framework includes Sparse Topical Coding, which is responsible for topic mining, and K-means that discovers the latent clusters in documents... 

Feature selection is a fundamental problem in machine learning and data mining. The majority of feature selection algorithms are designed for running on a single machine (centralized setting) and they are less applicable to very large datasets. Although there are some distributed methods to tackle this problem, most of them are distributing the data horizontally which are not suitable for datasets with a large number of features and few number of instances. Thus, in this paper, we introduce a novel vertically distributable feature selection method in order to speed up this process and be able to handle very large datasets in a scalable manner. In general, feature selection methods aim at... 

In recent years, end-to-end approaches to automatic speech recognition have received considerable attention as they are much faster in terms of preparing resources. However, conventional multistage approaches, which rely on a pipeline of training hidden Markov models (HMM)-GMM models and tree-building steps still give the state-of-the-art results on most databases. In this study, we investigate flat-start one-stage training of neural networks using lattice-free maximum mutual information (LF-MMI) objective function with HMM for large vocabulary continuous speech recognition. We thoroughly look into different issues that arise in such a setup and propose a standalone system, which achieves...