Can I read Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer on EtoBox?

Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer by Luo, Anwei; Cai, Rizhao; Kong, Chenqi; Ju, Yakun; Kang, Xiangui; Huang, Jiwu; Kot, Alex C. is a scholarly article available to read on EtoBox.

What is Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer about?

With the rapid progress of generative models, the current challenge in face forgery detection is how to effectively detect realistic manipulated faces from different unseen domains. Though previous studies show that pre-trained Vision Transformer (ViT) based models can achieve some promising results after fully fine-tuning on the Deepfake dataset, their generalization performances are still unsatisfactory. One possible reason is that fully fine-tuned ViT-based models may disrupt the pre-trained features [1, 2] and overfit to some data-specific patterns [3]. To alleviate this issue, we present a \textbf{F}orgery-aware \textbf{A}daptive \textbf{Vi}sion \textbf{T}ransformer (FA-ViT) under the adaptive learning paradigm, where the parameters in the pre-trained ViT are kept fixed while the designed adaptive modules are optimized to capture forgery features. Specifically, a global adaptive module is designed to model long-range interactions among input tokens, which takes advantage of self-attention mechanism to mine global forgery clues. To further explore essential local forgery clues, a local adaptive module is proposed to expose local inconsistencies by enhancing the local contextual

Author

Luo, Anwei; Cai, Rizhao; Kong, Chenqi; Ju, Yakun; Kang, Xiangui; Huang, Jiwu; Kot, Alex C.

Published

2023

Language

EN