Can I read Norm-guided Latent Space Exploration for Text-to-image Generation on EtoBox?

Norm-guided Latent Space Exploration for Text-to-image Generation by Samuel, Dvir; Ben-Ari, Rami; Darshan, Nir; Maron, Haggai; Chechik, Gal is a scholarly article available to read on EtoBox.

What is Norm-guided Latent Space Exploration for Text-to-image Generation about?

Text-to-image diffusion models show great potential in synthesizing a large variety of concepts in new compositions and scenarios. However, the latent space of initial seeds is still not well understood and its structure was shown to impact the generation of various concepts. Specifically, simple operations like interpolation and finding the centroid of a set of seeds perform poorly when using standard Euclidean or spherical metrics in the latent space. This paper makes the observation that, in current training procedures, diffusion models observed inputs with a narrow range of norm values. This has strong implications for methods that rely on seed manipulation for image generation, with applications to few-shot and long-tail learning tasks. To address this issue, we propose a novel method for interpolating between two seeds and demonstrate that it defines a new non-Euclidean metric that takes into account a norm-based prior on seeds. We describe a simple yet efficient algorithm for approximating this interpolation procedure and use it to further define centroids in the latent seed space. We show that our new interpolation and centroid techniques significantly enhance the generatio

Author

Samuel, Dvir; Ben-Ari, Rami; Darshan, Nir; Maron, Haggai; Chechik, Gal

Published

2023

Language

EN