Recent papers have demonstrated the possibility of energy-based text generation by adapting MCMC sampling algorithms that make use of gradient information to achieve faster convergence. However, as we show in this paper, previous attempts to apply this paradigm to text generation all fail to sample correctly from the target distributions. To address this limitation, we consider the problem of designing text samplers that are faithful, meaning that they converge to the target distribution---an energy-based language model. We propose several faithful gradient-based sampling algorithms, and study their theoretical properties. Through experiments on various forms of text generation, we demonstrate that faithful samplers are able to generate more fluent text while adhering to the control objectives better.