Can I read Can Pre-trained Networks Detect Familiar Out-of-Distribution Data? on EtoBox?

Can Pre-trained Networks Detect Familiar Out-of-Distribution Data? by Miyai, Atsuyuki; Yu, Qing; Irie, Go; Aizawa, Kiyoharu is a scholarly article available to read on EtoBox.

What is Can Pre-trained Networks Detect Familiar Out-of-Distribution Data? about?

Out-of-distribution (OOD) detection is critical for safety-sensitive machine learning applications and has been extensively studied, yielding a plethora of methods developed in the literature. However, most studies for OOD detection did not use pre-trained models and trained a backbone from scratch. In recent years, transferring knowledge from large pre-trained models to downstream tasks by lightweight tuning has become mainstream for training in-distribution (ID) classifiers. To bridge the gap between the practice of OOD detection and current classifiers, the unique and crucial problem is that the samples whose information networks know often come as OOD input. We consider that such data may significantly affect the performance of large pre-trained networks because the discriminability of these OOD data depends on the pre-training algorithm. Here, we define such OOD data as PT-OOD (Pre-Trained OOD) data. In this paper, we aim to reveal the effect of PT-OOD on the OOD detection performance of pre-trained networks from the perspective of pre-training algorithms. To achieve this, we explore the PT-OOD detection performance of supervised and self-supervised pre-training algorithms wit

Author

Miyai, Atsuyuki; Yu, Qing; Irie, Go; Aizawa, Kiyoharu

Published

2023

Language

EN