Can I read (Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Models on EtoBox?

(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Models by Kirsch, Andreas is a scholarly article available to read on EtoBox.

What is (Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Models about?

Epistemic uncertainty is crucial for safety-critical applications and out-of-distribution detection tasks. Yet, we uncover a paradoxical phenomenon in deep learning models: an epistemic uncertainty collapse as model complexity increases, challenging the assumption that larger models invariably offer better uncertainty quantification. We propose that this stems from implicit ensembling within large models. To support this hypothesis, we demonstrate epistemic uncertainty collapse empirically across various architectures, from explicit ensembles of ensembles and simple MLPs to state-of-the-art vision models, including ResNets and Vision Transformers -- for the latter, we examine implicit ensemble extraction and decompose larger models into diverse sub-models, recovering epistemic uncertainty. We provide theoretical justification for these phenomena and explore their implications for uncertainty estimation.

Author

Kirsch, Andreas

Published

2024

Language

EN