Can I read UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interface on EtoBox?

UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interface by Tang, Hao; Xie, Chenwei; Wang, Haiyang; Bao, Xiaoyi; Weng, Tingyu; Li, Pandeng; Zheng, Yun; Wang, Liwei is a scholarly article available to read on EtoBox.

What is UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interface about?

Generalist models have achieved remarkable success in both language and vision-language tasks, showcasing the potential of unified modeling. However, effectively integrating fine-grained perception tasks like detection and segmentation into these models remains a significant challenge. This is primarily because these tasks often rely heavily on task-specific designs and architectures that can complicate the modeling process. To address this challenge, we present \ours, a framework that \textbf{U}nifies \textbf{F}ine-grained visual perception tasks through an \textbf{O}pen-ended language interface. By transforming all perception targets into the language space, \ours unifies object-level detection, pixel-level segmentation, and image-level vision-language tasks into a single model. Additionally, we introduce a novel embedding retrieval approach that relies solely on the language interface to support segmentation tasks. Our framework bridges the gap between fine-grained perception and vision-language tasks, significantly simplifying architectural design and training strategies while achieving comparable or superior performance to methods with intricate task-specific designs. After mu

Author

Tang, Hao; Xie, Chenwei; Wang, Haiyang; Bao, Xiaoyi; Weng, Tingyu; Li, Pandeng; Zheng, Yun; Wang, Liwei

Published

2025

Language

EN