Astrophysical surveys present the challenge of scaling up accurate simulation based inference to billions of different examples. We develop a method to train fast, accurate and amortised approximate posteriors that avoids the biases of e.g. variational inference. To train our approximate posterior, we first sample from it, then we do a few steps of an MCMC method (we use HMC), then we update the approximate posterior parameters to maximize the probability of the resulting MCMC samples. This allows us to amortise the posterior implied by any MCMC procedure. On our astrophysical samples, the amortised approximate posterior is very close to the true MCMC posterior, yet is approximately five orders of magnitude faster.