Can I read FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise on EtoBox?

FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise by Yuan, Yunlong; Guo, Yuanfan; Wang, Chunwei; Zhang, Wei; Xu, Hang; Zhang, Li is a scholarly article available to read on EtoBox.

What is FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise about?

Text-driven video generation has advanced significantly due to developments in diffusion models. Beyond the training and sampling phases, recent studies have investigated noise priors of diffusion models, as improved noise priors yield better generation results. One recent approach employs the Fourier transform to manipulate noise, marking the initial exploration of frequency operations in this context. However, it often generates videos that lack motion dynamics and imaging details. In this work, we provide a comprehensive theoretical analysis of the variance decay issue present in existing methods, contributing to the loss of details and motion dynamics. Recognizing the critical impact of noise distribution on generation quality, we introduce FreqPrior, a novel noise initialization strategy that refines noise in the frequency domain. Our method features a novel filtering technique designed to address different frequency signals while maintaining the noise prior distribution that closely approximates a standard Gaussian distribution. Additionally, we propose a partial sampling process by perturbing the latent at an intermediate timestep during finding the noise prior, significantl

Author

Yuan, Yunlong; Guo, Yuanfan; Wang, Chunwei; Zhang, Wei; Xu, Hang; Zhang, Li

Published

2025

Language

EN