Can I read HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimation on EtoBox?

HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimation by Liu, Yibo; Jiang, Zhaodong; Xu, Binbin; Wu, Guile; Ren, Yuan; Cao, Tongtong; Liu, Bingbing; Yang, Rui Heng; Rasouli, Amir; Shan, Jinjun is a scholarly article available to read on EtoBox.

What is HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimation about?

This work focuses on model-free zero-shot 6D object pose estimation for robotics applications. While existing methods can estimate the precise 6D pose of objects, they heavily rely on curated CAD models or reference images, the preparation of which is a time-consuming and labor-intensive process. Moreover, in real-world scenarios, 3D models or reference images may not be available in advance and instant robot reaction is desired. In this work, we propose a novel framework named HIPPo, which eliminates the need for curated CAD models and reference images by harnessing image-to-3D priors from Diffusion Models, enabling model-free zero-shot 6D pose estimation. Specifically, we construct HIPPo Dreamer, a rapid image-to-mesh model built on a multiview Diffusion Model and a 3D reconstruction foundation model. Our HIPPo Dreamer can generate a 3D mesh of any unseen objects from a single glance in just a few seconds. Then, as more observations are acquired, we propose to continuously refine the diffusion prior mesh model by joint optimization of object geometry and appearance. This is achieved by a measurement-guided scheme that gradually replaces the plausible diffusion priors with more re

Author

Liu, Yibo; Jiang, Zhaodong; Xu, Binbin; Wu, Guile; Ren, Yuan; Cao, Tongtong; Liu, Bingbing; Yang, Rui Heng; Rasouli, Amir; Shan, Jinjun

Published

2025

Language

EN