Can I read Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference on EtoBox?

Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference by Gao, Yuan; Luo, Yajing; Wang, Junhong; Jia, Kui; Xia, Gui-Song is a scholarly article available to read on EtoBox.

What is Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference about?

Humans can easily deduce the relative pose of an unseen object, without label/training, given only a single query-reference image pair. This is arguably achieved by incorporating (i) 3D/2.5D shape perception from a single image, (ii) render-and-compare simulation, and (iii) rich semantic cue awareness to furnish (coarse) reference-query correspondence. Existing methods implement (i) by a 3D CAD model or well-calibrated multiple images and (ii) by training a network on specific objects, which necessitate laborious ground-truth labeling and tedious training, potentially leading to challenges in generalization. Moreover, (iii) was less exploited in the paradigm of (ii), despite that the coarse correspondence from (iii) enhances the compare process by filtering out non-overlapped parts under substantial pose differences/occlusions. Motivated by this, we propose a novel 3D generalizable relative pose estimation method by elaborating (i) with a 2.5D shape from an RGB-D reference, (ii) with an off-the-shelf differentiable renderer, and (iii) with semantic cues from a pretrained model like DINOv2. Specifically, our differentiable renderer takes the 2.5D rotatable mesh textured by the RGB a

Author

Gao, Yuan; Luo, Yajing; Wang, Junhong; Jia, Kui; Xia, Gui-Song

Published

2024

Language

EN