This paper investigates benchmarking and improving generative data augmentation. Generative data augmentation leverages the synthetic samples produced by generative models as an additional dataset for classification with small dataset settings. However, through benchmarking on multiple datasets, we found that generative data augmentation fails to boost performance when training dataset is small even if using state-of-the-art generative models. This is because the synthetic samples do not perfectly represent class categories in real data and uniform sampling does not necessarily provide useful samples for tasks. In this paper, we present a novel strategy for generative data augmentation called meta generative regularization (MGR). To avoid the degradation of generative data augmentation, MGR utilizes synthetic samples in the regularization term for feature extractors instead of in the loss function, e.g., cross-entropy. These synthetic samples are dynamically determined to minimize the validation losses through meta-learning. We observed that MGR can avoid the performance degradation of na\"ive generative data augmentation and boost the baselines. Experiments on six datasets showed that MGR is effective particularly when datasets are smaller and stably outperforms baselines.