Can I read Prompt Mixing in Diffusion Models using the Black Scholes Algorithm on EtoBox?

Prompt Mixing in Diffusion Models using the Black Scholes Algorithm by Kothandaraman, Divya; Lin, Ming; Manocha, Dinesh is a scholarly article available to read on EtoBox.

What is Prompt Mixing in Diffusion Models using the Black Scholes Algorithm about?

We introduce a novel approach for prompt mixing, aiming to generate images at the intersection of multiple text prompts using pre-trained text-to-image diffusion models. At each time step during diffusion denoising, our algorithm forecasts predictions w.r.t. the generated image and makes informed text conditioning decisions. To do so, we leverage the connection between diffusion models (rooted in non-equilibrium thermodynamics) and the Black-Scholes model for pricing options in Finance, and draw analogies between the variables in both contexts to derive an appropriate algorithm for prompt mixing using the Black Scholes model. Specifically, the parallels between diffusion models and the Black-Scholes model enable us to leverage properties related to the dynamics of the Markovian model derived in the Black-Scholes algorithm. Our prompt-mixing algorithm is data-efficient, meaning it does not need additional training. Furthermore, it operates without human intervention or hyperparameter tuning. We highlight the benefits of our approach by comparing it qualitatively and quantitatively to other prompt mixing techniques, including linear interpolation, alternating prompts, step-wise promp

Author

Kothandaraman, Divya; Lin, Ming; Manocha, Dinesh

Published

2024

Language

EN