Realistic synthetic time series data of sufficient length enables practical applications in time series modeling tasks, such as forecasting, but remains to be a challenge. In this paper we present PSA-GAN, a generative adversarial network (GAN) that generates long time series samples of high quality using progressive growing of GANs and self-attention. We show that PSA-GAN can be used to reduce the error in two downstream forecasting tasks over baselines that only use real data. We also introduce a Frechet-Inception Distance-like score, Context-FID, assessing the quality of synthetic time series samples. In our downstream tasks, we find that this score is able to predict the best-performing models and could therefore be a useful tool to develop time series GAN models for downstream use.