Loading...

Leveraging multi-modal fusion for graph-based image annotation

Amiri, S. H ; Sharif University of Technology | 2018

556 Viewed
  1. Type of Document: Article
  2. DOI: 10.1016/j.jvcir.2018.08.012
  3. Publisher: Academic Press Inc , 2018
  4. Abstract:
  5. Considering each of the visual features as one modality in image annotation task, efficient fusion of different modalities is essential in graph-based learning. Traditional graph-based methods consider one node for each image and combine its visual features into a single descriptor before constructing the graph. In this paper, we propose an approach that constructs a subgraph for each modality in such a way that edges of subgraph are determined using a search-based approach that handles class-imbalance challenge in the annotation datasets. Multiple subgraphs are then connected to each other to have a supergraph. This follows by introducing a learning framework to infer the tags of unannotated images on the supergraph. The proposed approach takes advantages of graph-based semi-supervised learning and multi-modal representation simultaneously. We evaluate the performance of the proposed approach on different datasets. The results reveal that the proposed approach improves the accuracy of annotation systems. © 2018 Elsevier Inc
  6. Keywords:
  7. Graph-based learning ; Image annotation ; Multi-modal representation ; Supergraph ; Tag ; Graphic methods ; Image analysis ; Image fusion ; Supervised learning ; Annotation systems ; Graph-based methods ; Graph-based semi-supervised learning ; Learning frameworks ; Manifold ; Multi-modal
  8. Source: Journal of Visual Communication and Image Representation ; Volume 55 , 2018 , Pages 816-828 ; 10473203 (ISSN)
  9. URL: https://www.sciencedirect.com/science/article/pii/S1047320318302037