Can I read ACT2G: Attention-based Contrastive Learning for Text-to-Gesture Generation on EtoBox?

ACT2G: Attention-based Contrastive Learning for Text-to-Gesture Generation by Teshima, Hitoshi; Wake, Naoki; Thomas, Diego; Nakashima, Yuta; Kawasaki, Hiroshi; Ikeuchi, Katsushi is a scholarly article available to read on EtoBox.

What is ACT2G: Attention-based Contrastive Learning for Text-to-Gesture Generation about?

Recent increase of remote-work, online meeting and tele-operation task makes people find that gesture for avatars and communication robots is more important than we have thought. It is one of the key factors to achieve smooth and natural communication between humans and AI systems and has been intensively researched. Current gesture generation methods are mostly based on deep neural network using text, audio and other information as the input, however, they generate gestures mainly based on audio, which is called a beat gesture. Although the ratio of the beat gesture is more than 70% of actual human gestures, content based gestures sometimes play an important role to make avatars more realistic and human-like. In this paper, we propose a attention-based contrastive learning for text-to-gesture (ACT2G), where generated gestures represent content of the text by estimating attention weight for each word from the input text. In the method, since text and gesture features calculated by the attention weight are mapped to the same latent space by contrastive learning, once text is given as input, the network outputs a feature vector which can be used to generate gestures related to the co

Author

Teshima, Hitoshi; Wake, Naoki; Thomas, Diego; Nakashima, Yuta; Kawasaki, Hiroshi; Ikeuchi, Katsushi

Published

2023

Language

EN