Can I read QORT-Former: Query-optimized Real-time Transformer for Understanding Two Hands Manipulating Objects on EtoBox?

QORT-Former: Query-optimized Real-time Transformer for Understanding Two Hands Manipulating Objects by Ismayilzada, Elkhan; Sayem, MD Khalequzzaman Chowdhury; Tiruneh, Yihalem Yimolal; Chowdhury, Mubarrat Tajoar; Boboev, Muhammadjon; Baek, Seungryul is a scholarly article available to read on EtoBox.

What is QORT-Former: Query-optimized Real-time Transformer for Understanding Two Hands Manipulating Objects about?

Significant advancements have been achieved in the realm of understanding poses and interactions of two hands manipulating an object. The emergence of augmented reality (AR) and virtual reality (VR) technologies has heightened the demand for real-time performance in these applications. However, current state-of-the-art models often exhibit promising results at the expense of substantial computational overhead. In this paper, we present a query-optimized real-time Transformer (QORT-Former), the first Transformer-based real-time framework for 3D pose estimation of two hands and an object. We first limit the number of queries and decoders to meet the efficiency requirement. Given limited number of queries and decoders, we propose to optimize queries which are taken as input to the Transformer decoder, to secure better accuracy: (1) we propose to divide queries into three types (a left hand query, a right hand query and an object query) and enhance query features (2) by using the contact information between hands and an object and (3) by using three-step update of enhanced image and query features with respect to one another. With proposed methods, we achieved real-time pose estimation

Author

Ismayilzada, Elkhan; Sayem, MD Khalequzzaman Chowdhury; Tiruneh, Yihalem Yimolal; Chowdhury, Mubarrat Tajoar; Boboev, Muhammadjon; Baek, Seungryul

Published

2025

Language

EN