Sharif Digital Repository

Image registration is an important task in medical image processing. Assuming spatially stationary intensity relation among images, conventional area based algorithms such as CC (Correlation Coefficients) and MI (Mutual Information), show weaker results alongside spatially varying intensity distortion. In this research, a structural representation of images is introduced. It allows us to use simpler similarity metrics in multimodal images which are also robust against the mentioned distortion field. The efficiency of this presentation in non-rigid image registration in the presence of spatial varying distortion field is examined. Experimental results on synthetic and real-world data sets... 

The problem of secure communication over multiple-Access Wiretap channel (MAC-WTC) under strong secrecy criterion is investigated. A new technique based on channel output statistics approximation is developed for establishing the strong security over multi-user channels. In particular, this technique shows that how simple wiretap coding results in secure communication under strong secrecy criterion instead of weak secrecy criterion. As a side result of the paper, two results on the output statistics of MAC are provided. Such results can be used to approximate the mutual information between input and output of MAC with respect to a given codebook of arbitrary rate  

Precise acoustical leak detection calls for robust time-delay estimates, which minimize the probability of false alarms in the face of dispersive propagation, multiple reflections, and uncorrelated background noise. Providing evidence that higher order modes and multi-reflected signals behave like sets of correlated noise, this work uses a regression model to optimize residual complexity in the presence of both correlated and uncorrelated noise. This optimized residual complexity (ORC) is highly robust since it takes into account both the level and complexity of noise. The lower complexity of the dispersive modes and multiple reflections, compared to the complexity of the plane mode, points... 

In CHES 2008 a generic side-channel distinguisher, Mutual Information, has been introduced to be independent of the relation between measurements and leakages as well as between leakages and data processed. Assuming a Gaussian model for the side-channel leakages, correlation power analysis (CPA) is capable of revealing the secrets efficiently. The goal of this paper is to compare mutual information analysis (MIA) and CPA when leakage of the target device fits into a Gaussian assumption. We first theoretically examine why MIA can reveal the correct key guess amongst other hypotheses, and then compare it with CPA proofs. As our theoretical comparison confirms and shown recently in ACNS 2009... 

In this paper an approach for blind source separation in bilinear (or linear quadratic) mixtures is presented. The proposed algorithm employs the same recurrent structure as [Hosseini and Deville, 2003) for separating these mixtures . However, instead of maximal likelihood, our algorithm is based on minimizing the mutual information of the outputs for recovering the independent components. Simulation results show the efficiency of the proposed algorithm. © 2009 IEEE  

Theorem 5 of A. Gohari, V. Anantharam, IEEE Transactions on Information Theory, vol. 56, no. 8, pp. 3973-3996, 2010, states an upper bound on the secrecy capacity for the source model problem. It has a three page proof given in Appendix B of the paper. Unfortunately, we show that this bound does not provide any improvement over the simpler bound given in Corollary 1 of the paper. We also provide an example of a family of two agent source model problems where the one-way secrecy rate in each direction is zero, but the secrecy rate is nonzero and can be determined exactly as a conditional mutual information. © 1963-2012 IEEE  

Due to unstoppable growth of social networks and the large number of users, the detection of communities have become one of the most popular and successful domain of research areas. Detecting communities is a significant aspect in analyzing networks because of its various applications such as sampling, link prediction and communications among members of social networks. There have been proposed many different algorithms for solving community detection problem containing optimization methods. In this paper we propose a novel algorithm based on extended distributed learning automata for solving this problem. Our proposed algorithm benefits from cooperation between learning automata to detect... 

In this paper, the impact of a small variations in the channel gains on the secrecy rate of the wiretap channel is studied, in which it is assumed that the imperfect channel knowledge is available at the transmitter. First, we consider general additive noise model for both legitimate and eavesdropper channels in the wiretap channel, and compute the variation of the secrecy rate resulting from the small variations in the channel gains. Then, we focus on the Gaussian wiretap channel, as a special case and calculate the sensitivity of the secrecy capacity to the channel gains with imperfect channel knowledge. Interestingly, it is shown that in some situations the effect of the channel variation... 

In this paper, we introduce a document clustering method based on Sparse Topical Coding, called Cluster-based Sparse Topical Coding. Topic modeling is capable of improving textual document clustering by describing documents via bag-of-words models and projecting them into a topic space. The latent semantic descriptions derived by the topic model can be utilized as features in a clustering process. In our proposed method, document clustering and topic modeling are integrated in a unified framework in order to achieve the highest performance. This framework includes Sparse Topical Coding, which is responsible for topic mining, and K-means that discovers the latent clusters in documents... 

Feature selection is a fundamental problem in machine learning and data mining. The majority of feature selection algorithms are designed for running on a single machine (centralized setting) and they are less applicable to very large datasets. Although there are some distributed methods to tackle this problem, most of them are distributing the data horizontally which are not suitable for datasets with a large number of features and few number of instances. Thus, in this paper, we introduce a novel vertically distributable feature selection method in order to speed up this process and be able to handle very large datasets in a scalable manner. In general, feature selection methods aim at... 

In recent years, end-to-end approaches to automatic speech recognition have received considerable attention as they are much faster in terms of preparing resources. However, conventional multistage approaches, which rely on a pipeline of training hidden Markov models (HMM)-GMM models and tree-building steps still give the state-of-the-art results on most databases. In this study, we investigate flat-start one-stage training of neural networks using lattice-free maximum mutual information (LF-MMI) objective function with HMM for large vocabulary continuous speech recognition. We thoroughly look into different issues that arise in such a setup and propose a standalone system, which achieves... 

In this paper, we introduce a document clustering method based on Sparse Topical Coding, called Cluster-based Sparse Topical Coding. Topic modeling is capable of improving textual document clustering by describing documents via bag-of-words models and projecting them into a topic space. The latent semantic descriptions derived by the topic model can be utilized as features in a clustering process. In our proposed method, document clustering and topic modeling are integrated in a unified framework in order to achieve the highest performance. This framework includes Sparse Topical Coding, which is responsible for topic mining, and K-means that discovers the latent clusters in documents... 

We present our work on end-to-end training of acoustic models using the lattice-free maximum mutual information (LF-MMI) objective function in the context of hidden Markov models. By end-to-end training, we mean flat-start training of a single DNN in one stage without using any previously trained models, forced alignments, or building state-tying decision trees. We use full biphones to enable context-dependent modeling without trees, and show that our end-to-end LF-MMI approach can achieve comparable results to regular LF-MMI on well-known large vocabulary tasks. We also compare with other end-to-end methods such as CTC in character-based and lexicon-free settings and show 5 to 25 percent... 

Similarity measure is a main core of image registration algorithms. Spatially varying intensity distortion is an important challenge, which affects the performance of similarity measures. Correlation among the pixels is the main characteristic of this distortion. Similarity measures such as sum-of-squared-differences (SSD) and mutual information ignore this correlation; hence, perfect registration cannot be achieved in the presence of this distortion. In this paper, we model this correlation with the aid of the low rank matrix theory. Based on this model, we compensate this distortion analytically and introduce rank-regularized SSD (RRSSD). This new similarity measure is a modified SSD based... 

A measure of correlation is said to have the tensorization property if it does not change when computed for i.i.d. copies. More precisely, a measure of correlation between two random variables X, Y denoted by rho (X, Y), has the tensorization property if ρ(Xn, Yn)=ρ (X, Y) where (Xn, Yn) denotes n i.i.d. copies of (X, Y). Two well-known examples of such measures are the maximal correlation and the hypercontractivity ribbon (HC ribbon). We show that the maximal correlation and the HC ribbon are special cases of the new notion of Φ-ribbons, defined in this paper for a class of convex functions Φ. Φ-ribbon reduces to the HC ribbon and the maximal correlation for special choices of Φ, and is a... 

A video activity analysis is presented based on 3D wavelet transform. Marginal and joint statistics as well as mutual information estimates are extracted. Marginal histograms are approximated by Generalized Gaussian Density (GGD) functions. The mutual information between coefficients -as a quantitative estimate of joint statistics- decreases when the activity in the video increases. The relationship between kurtosis graphs, extracted from joint distributions and video activity, is deduced. Results show that the type of activity in the video can be figured out from Kurtosis curves. The GGD and the Kullback-Leibler distance (KLD) are used to retrieve and locate 96% of videos properly  

In this paper, a new measure of correlation is introduced. This measure depends on a parameter α, and is defined in terms of vector-valued Lp norms. The measure is within a constant of the exponential of α-Rényi mutual information, and reduces to the trace norm (total variation distance) for α = 1. We provide some properties and applications of this measure of correlation. In particular, we establish a bound on the secrecy exponent of the wiretap channel (under the total variation metric) in terms of the α-Rényi mutual information according to Csiszár's proposal. © 2019 IEEE  

We present our work on improving the numerator graph for discriminative training using the lattice-free maximum mutual information (MMI) criterion. Specifically, we propose a scheme for creating unconstrained numerator graphs by removing time constraints from the baseline numerator graphs. This leads to much smaller graphs and therefore faster preparation of training supervisions. By testing the proposed un-constrained supervisions using factorized time-delay neural network (TDNN) models, we observe 0.5% to 2.6% relative improvement over the state-of-the-art word error rates on various large-vocabulary speech recognition databases. © 2018 IEEE  

Correlations disguised in various forms underlie a host of important phenomena in classical and quantum systems, such as information and energy exchanges. The quantum mutual information and the norm of the correlation matrix are both considered as proper measures of total correlations. We demonstrate that, when applied to the same system, these two measures can actually show significantly different behavior except at least in two limiting cases: when there are no correlations and when there is maximal quantum entanglement. We further quantify the discrepancy by providing analytic formulas for time derivatives of the measures for an interacting bipartite system evolving unitarily. We argue... 

Blind source separation (BSS) techniques have revealed to be promising approaches for the noninvasive extraction of fetal cardiac signals from maternal abdominal recordings. From previous studies, it is now believed that a carefully selected array of electrodes well-placed over the abdomen of a pregnant woman contains the required 'information' for BSS, to extract the complete fetal components. Based on this idea, previous works have involved array recording systems and sensor selection strategies based on the Mutual Information (MI) criterion. In this paper the previous works have been extended, by considering the 3-dimensional aspects of the cardiac electrical activity. The proposed method...