Can I read TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up on EtoBox?

TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up by Jiang, Yifan; Chang, Shiyu; Wang, Zhangyang is a scholarly article available to read on EtoBox.

What is TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up about?

The recent explosive interest on transformers has suggested their potential to become powerful "universal" models for computer vision tasks, such as classification, detection, and segmentation. While those attempts mainly study the discriminative models, we explore transformers on some more notoriously difficult vision tasks, e.g., generative adversarial networks (GANs). Our goal is to conduct the first pilot study in building a GAN completely free of convolutions, using only pure transformer-based architectures. Our vanilla GAN architecture, dubbed TransGAN, consists of a memory-friendly transformer-based generator that progressively increases feature resolution, and correspondingly a multi-scale discriminator to capture simultaneously semantic contexts and low-level textures. On top of them, we introduce the new module of grid self-attention for alleviating the memory bottleneck further, in order to scale up TransGAN to high-resolution generation. We also develop a unique training recipe including a series of techniques that can mitigate the training instability issues of TransGAN, such as data augmentation, modified normalization, and relative position encoding. Our best archite

Author

Jiang, Yifan; Chang, Shiyu; Wang, Zhangyang

Published

2021

Language

EN