Can I read LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control on EtoBox?

LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control by Guo, Jianzhu; Zhang, Dingyun; Liu, Xiaoqiang; Zhong, Zhizhou; Zhang, Yuan; Wan, Pengfei; Zhang, Di is a scholarly article available to read on EtoBox.

What is LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control about?

Portrait Animation aims to synthesize a lifelike video from a single source image, using it as an appearance reference, with motion (i.e., facial expressions and head pose) derived from a driving video, audio, text, or generation. Instead of following mainstream diffusion-based methods, we explore and extend the potential of the implicit-keypoint-based framework, which effectively balances computational efficiency and controllability. Building upon this, we develop a video-driven portrait animation framework named LivePortrait with a focus on better generalization, controllability, and efficiency for practical usage. To enhance the generation quality and generalization ability, we scale up the training data to about 69 million high-quality frames, adopt a mixed image-video training strategy, upgrade the network architecture, and design better motion transformation and optimization objectives. Additionally, we discover that compact implicit keypoints can effectively represent a kind of blendshapes and meticulously propose a stitching and two retargeting modules, which utilize a small MLP with negligible computational overhead, to enhance the controllability. Experimental results dem

Author

Guo, Jianzhu; Zhang, Dingyun; Liu, Xiaoqiang; Zhong, Zhizhou; Zhang, Yuan; Wan, Pengfei; Zhang, Di

Published

2024

Language

EN