Can I read Referring Expression Comprehension by Composing Semantic-based Visual Attention on EtoBox?

Referring Expression Comprehension by Composing Semantic-based Visual Attention by Zheng-An Zhu; Hsuan-Lun Chiang; Chen-Kuo Chiang is a scholarly article available to read on EtoBox.

What is Referring Expression Comprehension by Composing Semantic-based Visual Attention about?

In recent years, multimodal tasks are receiving more and more attentions. Referring expression comprehension aims to find the object by a natural language sentence. The challenge of this task is that the system should be able to recognize images and understand the text to determine the corresponding target. In this paper, an interpretable method is proposed to perform referring expression comprehension by decomposing the input sentences into semantic units. The proposed method learns the joint text and visual features and generates the attention map of the target object. Visualization of comprehension progress can be obtained in the end to further analyze how deep model interpret the query object.

Author

Zheng-An Zhu; Hsuan-Lun Chiang; Chen-Kuo Chiang

Publisher

IEEE

Published

2022

Language

EN