Sharif Digital Repository

Different types of visual features provide multi-modal representation for images in the annotation task. Conventional graph-based image annotation methods integrate various features into a single descriptor and consider one node for each descriptor on the learning graph. However, this graph does not capture the information of individual features, making it unsuitable for propagating the labels of annotated images. In this paper, we address this issue by proposing an approach for fusing the visual features such that a specific subgraph is constructed for each visual modality and then subgraphs are connected to form a supergraph. As the size of supergraph grows linearly with the number of... 

In this paper, the problem of multimodal soft coupling under the Bayesian framework when variance of probabilistic model is unknown is investigated. Similarity of shared factors resulted from Nonnegative Matrix Factorization (NMF) of multimodal data sets is controlled in a soft manner by using a probabilistic model. In previous works, it is supposed that the probabilistic model and its parameters are known. However, this assumption does not always hold. In this paper it is supposed that the probabilistic model is already known but its variance is unknown. So the proposed algorithm estimates the variance of the probabilistic model along with the other parameters during the factorization... 

Considering each of the visual features as one modality in image annotation task, efficient fusion of different modalities is essential in graph-based learning. Traditional graph-based methods consider one node for each image and combine its visual features into a single descriptor before constructing the graph. In this paper, we propose an approach that constructs a subgraph for each modality in such a way that edges of subgraph are determined using a search-based approach that handles class-imbalance challenge in the annotation datasets. Multiple subgraphs are then connected to each other to have a supergraph. This follows by introducing a learning framework to infer the tags of... 

Multi-modal data comprising imaging (MRI, fMRI, PET, etc.) and non-imaging (clinical test, demographics, etc.) data can be collected together and used for disease prediction. Such diverse data gives complementary information about the patient's condition to make an informed diagnosis. A model capable of leveraging the individuality of each multi-modal data is required for better disease prediction. We propose a graph convolution based deep model which takes into account the distinctiveness of each element of the multi-modal data. We incorporate a novel self-attention layer, which weights every element of the demographic data by exploring its relation to the underlying disease. We demonstrate... 

Cascaded Short Term Hydro-Thermal Scheduling problem (CSTHTS) is a single objective, non-linear multi-modal or convex (depending upon the cost function of thermal generation) type of Short Term Hydro-Thermal Scheduling (STHTS), having complex hydel constraints. It has been solved by many metaheuristic optimization algorithms, as found in the literature. Recently, the authors have published the best-achieved results of the CSTHTS problem having quadratic fuel cost function of thermal generation using an improved variant of the Accelerated PSO (APSO) algorithm, as compared to the other previously implemented algorithms. This article discusses and presents further improvement in the results... 

In many real-world applications (e.g. social media application), data usually consists of diverse input modalities that originates from various heterogeneous sources. Learning a similarity measure for such data is of great importance for vast number of applications such as classification, clustering, retrieval, etc. Defining an appropriate distance metric between data points with multiple modalities is a key challenge that has a great impact on the performance of many multimedia applications. Existing approaches for multi-modal distance metric learning only offer point estimation of the distance matrix and/or latent features, and can therefore be unreliable when the number of training... 

Short term hydrothermal scheduling (STHTS) is a non-linear, multi-modal and very complex constrained optimization problem which has been solved using several conventional and modern metaheuristic optimization algorithms. A number of research articles have been published addressing STHTS using different techniques. This article presents a comprehensive review of research published for solving the STHTS problem in the last four decades. © 2013 IEEE  

Deep learning has received much attention as of the most powerful approaches for multimodal representation learning in recent years. An ideal model for multimodal data can reason about missing modalities using the available ones, and usually provides more information when multiple modalities are being considered. All the previous deep models contain separate modality-specific networks and find a shared representation on top of those networks. Therefore, they only consider high level interactions between modalities to find a joint representation for them. In this paper, we propose a multimodal deep learning framework (MDLCW) that exploits the cross weights between representation of... 

Recently, the problem of integrating side information about classes has emerged in the learning settings like zero-shot learning. Although using multiple sources of information about the input space has been investigated in the last decade and many multi-view and multi-modal learning methods have already been introduced, the attribute learning for classes (output space) is a new problem that has been attended in the last few years. In this paper, we propose an attribute learning method that can use different sources of descriptions for classes to find new attributes that are more proper to be used as class signatures. Experimental results show that the learned attributes by the proposed... 

Estimation of distribution algorithms have evolved as a technique for estimating population distribution in evolutionary algorithms. They estimate the distribution of the candidate solutions and then sample the next generation from the estimated distribution. Bayesian optimization algorithm is an estimation of distribution algorithm, which uses a Bayesian network to estimate the distribution of candidate solutions and then generates the next generation by sampling from the constructed network. The experimental results show that the Bayesian optimization algorithms are capable of identifying correct linkage between the variables of optimization problems. Since the problem of finding the... 

In this paper, the problem of single microphone source separation via Nonnegative Matrix Factorization (NMF) by exploiting video information is addressed. Respective audio and video modalities coming from a single human speech usually have similar time changes. It means that changes in one of them usually corresponds to changes in the other one. So it is expected that activation coefficient matrices of their NMF decomposition are similar. Based on this similarity, in this paper the activation coefficient matrix of the video modality is used as an initialization for audio source separation via NMF. In addition, the mentioned similarity is used for post-processing and for clustering the rows...