MonoFlow: rethinking divergence GANs via the perspective of differential equations

Yi, Mingxuan, Zhu, Zhanxing and Liu, Song (2023) MonoFlow: rethinking divergence GANs via the perspective of differential equations. Krause, Andreas, Brunskill, Emma, Cho, Kyunghyun, Engelhardt, Barbara, Sabato, Sivan and Scarlett, Jonathan (eds.) In Proceedings of the 40th International Conference on Machine Learning (ICML). Association for Computing Machinery. 39984–40000 . (doi:10.5555/3618408.3620079).

Abstract

The conventional understanding of adversarial training in generative adversarial networks (GANs) is that the discriminator is trained to estimate a divergence, and the generator learns to minimize this divergence. We argue that despite the fact that many variants of GANs were developed following this paradigm, the current theoretical understanding of GANs and their practical algorithms are inconsistent. In this paper, we leverage Wasserstein gradient flows which characterize the evolution of particles in the sample space, to gain theoretical insights and algorithmic inspiration for GANs. We introduce a unified generative modeling framework – MonoFlow: the particle evolution is rescaled via a monotonically increasing mapping of the log density ratio. Under our framework, adversarial training can be viewed as a procedure first obtaining MonoFlow's vector field via training the discriminator and the generator learns to draw the particle flow defined by the corresponding vector field. We also reveal the fundamental difference between variational divergence minimization and adversarial training. This analysis helps us to identify what types of generator loss functions can lead to the successful training of GANs and suggest that GANs may have more loss designs beyond the literature (e.g., nonsaturated loss), as long as they realize MonoFlow. Consistent empirical studies are included to validate the effectiveness of our framework.

This record has no associated files available for download.

Contributors

Download statistics