Can I read Object-and-Action Aware Model for Visual Language Navigation on EtoBox?

Object-and-Action Aware Model for Visual Language Navigation by Qi, Yuankai; Pan, Zizheng; Zhang, Shengping; Hengel, Anton van den; Wu, Qi is a scholarly article available to read on EtoBox.

What is Object-and-Action Aware Model for Visual Language Navigation about?

Vision-and-Language Navigation (VLN) is unique in that it requires turning relatively general natural-language instructions into robot agent actions, on the basis of the visible environment. This requires to extract value from two very different types of natural-language information. The first is object description (e.g., 'table', 'door'), each presenting as a tip for the agent to determine the next action by finding the item visible in the environment, and the second is action specification (e.g., 'go straight', 'turn left') which allows the robot to directly predict the next movements without relying on visual perceptions. However, most existing methods pay few attention to distinguish these information from each other during instruction encoding and mix together the matching between textual object/action encoding and visual perception/orientation features of candidate viewpoints. In this paper, we propose an Object-and-Action Aware Model (OAAM) that processes these two different forms of natural language based instruction separately. This enables each process to match object-centered/action-centered instruction to their own counterpart visual perception/action orientation flexib

Author

Qi, Yuankai; Pan, Zizheng; Zhang, Shengping; Hengel, Anton van den; Wu, Qi

Published

2020

Language

EN