Can I read NaturalBench: Evaluating VLMs on EtoBox?

NaturalBench: Evaluating VLMs by Ahmed Anwar Mazher is a document available to read on EtoBox.

What is NaturalBench: Evaluating VLMs about?

The document introduces NaturalBench, a new benchmark designed to evaluate vision-language models (VLMs) on natural adversarial samples, which are questions about natural images that humans can easily answer but VLMs struggle with. It highlights the limitations of current VLMs, showing that even state-of-the-art models lag significantly behind human performance, and emphasizes the need for a vision-centric approach to prevent models from relying solely on language biases. The benchmark consists of 10,000 hu

Author

Ahmed Anwar Mazher

Language

EN