Loading...
Leveraging multi-modal fusion for graph-based image annotation
Amiri, S. H ; Sharif University of Technology | 2018
556
Viewed
- Type of Document: Article
- DOI: 10.1016/j.jvcir.2018.08.012
- Publisher: Academic Press Inc , 2018
- Abstract:
- Considering each of the visual features as one modality in image annotation task, efficient fusion of different modalities is essential in graph-based learning. Traditional graph-based methods consider one node for each image and combine its visual features into a single descriptor before constructing the graph. In this paper, we propose an approach that constructs a subgraph for each modality in such a way that edges of subgraph are determined using a search-based approach that handles class-imbalance challenge in the annotation datasets. Multiple subgraphs are then connected to each other to have a supergraph. This follows by introducing a learning framework to infer the tags of unannotated images on the supergraph. The proposed approach takes advantages of graph-based semi-supervised learning and multi-modal representation simultaneously. We evaluate the performance of the proposed approach on different datasets. The results reveal that the proposed approach improves the accuracy of annotation systems. © 2018 Elsevier Inc
- Keywords:
- Graph-based learning ; Image annotation ; Multi-modal representation ; Supergraph ; Tag ; Graphic methods ; Image analysis ; Image fusion ; Supervised learning ; Annotation systems ; Graph-based methods ; Graph-based semi-supervised learning ; Learning frameworks ; Manifold ; Multi-modal
- Source: Journal of Visual Communication and Image Representation ; Volume 55 , 2018 , Pages 816-828 ; 10473203 (ISSN)
- URL: https://www.sciencedirect.com/science/article/pii/S1047320318302037