We propose the novel framework for generative modeling using hybrid energy-based models. In our method we combine the interpretable input gradients of the robust classifier and Langevin Dynamics for sampling. Using the adversarial training we improves not only the training stability, but robustness and generative modelling of the joint energy-based models.