Can I read ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models on EtoBox?

ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models by Wang, Weijie; Ren, Wenqi; Mei, Guofeng; Ren, Bin; Huang, Xiaoshui; Poiesi, Fabio; Sebe, Nicu; Lepri, Bruno is a scholarly article available to read on EtoBox.

What is ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models about?

State-of-the-art 3D point cloud registration methods rely on labeled 3D datasets for training, which limits their practical applications in real-world scenarios and often hinders generalization to unseen scenes. Leveraging the zero-shot capabilities of foundation models offers a promising solution to these challenges. In this paper, we introduce ZeroReg, a zero-shot registration approach that utilizes 2D foundation models to predict 3D correspondences. Specifically, ZeroReg adopts an object-to-point matching strategy, starting with object localization and semantic feature extraction from multi-view images using foundation models. In the object matching stage, semantic features help identify correspondences between objects across views. However, relying solely on semantic features can lead to ambiguity, especially in scenes with multiple instances of the same category. To address this, we construct scene graphs to capture spatial relationships among objects and apply a graph matching algorithm to these graphs to accurately identify matched objects. Finally, computing fine-grained point-level correspondences within matched object regions using algorithms like SuperGlue and LoFTR achi

Author

Wang, Weijie; Ren, Wenqi; Mei, Guofeng; Ren, Bin; Huang, Xiaoshui; Poiesi, Fabio; Sebe, Nicu; Lepri, Bruno

Published

2023

Language

EN