Can I read Rethinking Vision Transformer Through Human–object Interaction Detection on EtoBox?

Rethinking Vision Transformer Through Human–object Interaction Detection by Yamin Cheng; Zitian Zhao; Zhi Wang; Hancong Duan is a Computer Science article available to read on EtoBox.

What is Rethinking Vision Transformer Through Human–object Interaction Detection about?

Recent works have shown that Vision Transformer models (ViT) can achieve comparable or even superior performance on image- and region-level recognition tasks, i.e., image recognition and object detection. However, can Vision Transformer perform region-level relationship reasoning with minimal information about the spatial geometry formation of input images? To answer this question, we propose the Region-level Relationship Reasoning Vision Transformer (R3ViT), a family of human–object interaction detection models based on the vanilla Vision Transformer with the fewest possible revisions, common region priors, as well as inductive biases of the objective task. Specifically, we first divide the input images into several local patches, replace the specialized [CLS ] token in vanilla ViT with extra relationship semantics carrier tokens in the entanglement-/pair-/triplet-wise manner and calculate both representations and their relevance. We assign each extra token with an individual supervision and compute the training loss in a dense manner. We find the vision transformer simply adjusted by the novel paradigm can already reason about the region-level visual relationship, e.g., R3ViT can

Who reads Rethinking Vision Transformer Through Human–object Interaction Detection?

Author

Yamin Cheng; Zitian Zhao; Zhi Wang; Hancong Duan

Publisher

Elsevier BV

Published

2023

Language

EN

Field

Computer Science (Physical Sciences)