Latent-Guided Cooperative Energy-Based Models

Energy-based models (EBMs) provide a flexible framework for generative models with strong distribution modeling capabilities. Nevertheless, their broader adoption has been limited by the difficulty of stable and efficient training. In this paper, we propose a unified and efficient latent-guided cooperative EBM that leverages informative target latent variables to guide the joint energy in capturing both data distribution and semantic structure, along with a cooperative generator designed for effective MCMC initialization. Our joint space optimization only requires MCMC sampling in the data space, and allows the energy to learn semantic data–latent relationships directly from real data. Experiments show our method improves generation quality and training stability with fewer resources, and performs effectively across multiple downstream tasks.

How can we model both the geometry and the distributional properties of data space? To address this question, we propose training a joint energy function. The key idea is to use informative pretrained latent representations to guide the learning of this energy function, together with a cooperative generator that makes training more effective. We find that this mechanism helps the joint energy function capture not only the overall data distribution, but also the semantic structure of the data manifold. As a result, it can improve both generation quality and out-of-distribution detection ability. This research provides a more intuitive way to understand the geometry and statistical properties of data space, and it also helps improve the generation capability of generative models.