Loading...

Automatic image annotation using tag relations and graph convolutional networks

Lotfi, F ; Sharif University of Technology | 2021

230 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/IPRIA53572.2021.9483536
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2021
  4. Abstract:
  5. Automatic image annotation is a mechanism to assign a list of appropriate tags that describe the visual content of a given image. Most methods only focus on the content of the images and ignore the relationship between the tags in vocabulary. In this work, we propose a new deep learning-based automatic image annotation architecture, which considers label dependencies in a graph convolution neural network structure and extracts tag descriptors to re-weight the output class scores based on their relationships. The proposed architecture has three main parts: feature extraction, graph convolutional network, and annotation. In graph convolutional network, we apply one layer convolution on vocabulary graph to get some tag descriptors that are applied on the image features. In the annotation part, we use two previous algorithms to rectify the tags. Compared to related work, the experimental results using Corel5k, Esp Game, and IAPRTC-12 datasets indicate the best performance of the annotating model with F1score in IAPRTC-12 and the second-best result with F1score and N+ measure in Corel5K and finally the second-best result with F1score in ESP Game. © 2021 IEEE
  6. Keywords:
  7. Convolution ; Convolutional neural networks ; Deep learning ; Image analysis ; Network architecture ; Pattern recognition ; Automatic image annotation ; Convolution neural network ; Convolutional networks ; Image features ; Label dependencies ; Proposed architectures ; Related works ; Visual content ; Image annotation
  8. Source: 5th International Conference on Pattern Recognition and Image Analysis, IPRIA 2021, 28 April 2021 through 29 April 2021 ; 2021 ; 9781665426596 (ISBN)
  9. URL: https://ieeexplore.ieee.org/document/9483536