Episodic replay methods, which store and replay past data, have been effective in handling distribution shifts in continual learning. However, due to regulatory and privacy concerns for data sharing, their applicability can be limited. In this work, we introduce two novel healthcare benchmarks for domain incremental continual learning: diabetic retinopathy severity classification and dermoscopy skin lesion detection, and highlight issues of poor forward and backward transferability in simple baselines. To overcome these challenges, we propose a novel method called prompt-based generative replay. By leveraging a text-to-image diffusion model for synthetic data generation, our approach effectively preserves previously learned knowledge while adapting to new data distributions. Our experiments demonstratethat our prompt-based generative replay significantly outperforms competitive baselines, resulting in an average increase of up to 5 points in average AUC for the skin lesions benchmark and up to 2 points for the diabetic retinopathy benchmark.