Can I read PartGlot: Learning Shape Part Segmentation from Language Reference Games on EtoBox?

PartGlot: Learning Shape Part Segmentation from Language Reference Games by Koo, Juil; Huang, Ian; Achlioptas, Panos; Guibas, Leonidas; Sung, Minhyuk is a scholarly article available to read on EtoBox.

What is PartGlot: Learning Shape Part Segmentation from Language Reference Games about?

We introduce PartGlot, a neural framework and associated architectures for learning semantic part segmentation of 3D shape geometry, based solely on part referential language. We exploit the fact that linguistic descriptions of a shape can provide priors on the shape's parts -- as natural language has evolved to reflect human perception of the compositional structure of objects, essential to their recognition and use. For training, we use the paired geometry / language data collected in the ShapeGlot work for their reference game, where a speaker creates an utterance to differentiate a target shape from two distractors and the listener has to find the target based on this utterance. Our network is designed to solve this target discrimination problem, carefully incorporating a Transformer-based attention module so that the output attention can precisely highlight the semantic part or parts described in the language. Furthermore, the network operates without any direct supervision on the 3D geometry itself. Surprisingly, we further demonstrate that the learned part information is generalizable to shape classes unseen during training. Our approach opens the possibility of learning 3D

Author

Koo, Juil; Huang, Ian; Achlioptas, Panos; Guibas, Leonidas; Sung, Minhyuk

Published

2021

Language

EN