Can I read Visual Representation Alignment for Multimodal Large Language Models on EtoBox?

Visual Representation Alignment for Multimodal Large Language Models by Heeji Yoon & Jaewoo Jung & Junwan Kim & Hyungyu Choi & Heeseong Shin & Sangbeom Lim & Honggyu An & Chaehyun Kim & Jisang Han & Donghyun Kim & Chanho Eom & Sunghwan Hong & Seungryong Kim is a book available to read on EtoBox.

What is Visual Representation Alignment for Multimodal Large Language Models about?

arXiv:2509.07979v1 [cs.CV] 9 Sep 2025ABSTRACTMultimodal large language models (MLLMs) trained with visual instruction tuning have achieved strong performance across diverse tasks, yet they remain limited in vision-centric tasks such as object counting or spatial reasoning. We attribute this gap to the prevailing text-only supervision paradigm, which providesonly indirect guidance for the visual pathway and often leads MLLMs to discardfine-grained visual details during training. In this paper, we present VIsual Representation ALignment (VIRAL), a simple yet effective regularization strategythat aligns the internal visual representations of MLLMs with those of pre-trainedvision foundation models (VFMs). By explicitly enforcing this alignment, VIRAL enables the model not only to retain critical visual details from the inputvision encoder but also to complement additional visual knowledge from VFMs,thereby enhancing its ability to reason over complex visual inputs. Our experiments demonstrate consistent improvements across all tasks on widely adoptedmultimodal benchmarks. Furthermore, we conduct comprehensive ablation studies to validate the key design choices underlying our fra

Author

Heeji Yoon & Jaewoo Jung & Junwan Kim & Hyungyu Choi & Heeseong Shin & Sangbeom Lim & Honggyu An & Chaehyun Kim & Jisang Han & Donghyun Kim & Chanho Eom & Sunghwan Hong & Seungryong Kim

Published

2025

Language

EN