Can I read ParNet: Position-aware Aggregated Relation Network for Image-Text matching on EtoBox?

ParNet: Position-aware Aggregated Relation Network for Image-Text matching by Xia, Yaxian; Huang, Lun; Wang, Wenmin; Wei, Xiaoyong; Wang, Wenmin is a scholarly article available to read on EtoBox.

What is ParNet: Position-aware Aggregated Relation Network for Image-Text matching about?

Exploring fine-grained relationship between entities(e.g. objects in image or words in sentence) has great contribution to understand multimedia content precisely. Previous attention mechanism employed in image-text matching either takes multiple self attention steps to gather correspondences or uses image objects (or words) as context to infer image-text similarity. However, they only take advantage of semantic information without considering that objects' relative position also contributes to image understanding. To this end, we introduce a novel position-aware relation module to model both the semantic and spatial relationship simultaneously for image-text matching in this paper. Given an image, our method utilizes the location of different objects to capture spatial relationship innovatively. With the combination of semantic and spatial relationship, it's easier to understand the content of different modalities (images and sentences) and capture fine-grained latent correspondences of image-text pairs. Besides, we employ a two-step aggregated relation module to capture interpretable alignment of image-text pairs. The first step, we call it intra-modal relation mechanism, in whic

Author

Xia, Yaxian; Huang, Lun; Wang, Wenmin; Wei, Xiaoyong; Wang, Wenmin

Published

2019

Language

EN