Episodic replay methods, which store and replay past data, have proven effective for handling distribution shifts in continual learning. However, due to regulatory and privacy concerns for data sharing, their applicability can be limited, especially in healthcare.In this work, we advance the state of art,focusing our inquiry on two novel benchmarks for domain incremental continual learning: diabetic retinopathy severity classification and dermoscopy skin lesion detection.First, we demonstrate the poor forward and backward transferability of simple baselines. Then, to overcome these challenges, we propose a novel method called conditional diffusion replay. By leveraging a text-to-image diffusion model for synthetic data generation, our approach effectively preservesperformance on previously encountered domainswhile adapting to new ones.We observe that compared to standard sequential fine-tuning, our conditional diffusion replay method improves average AUC by up to 6.2 points and 3.3 points for the skin lesions and diabetic retinopathy benchmarks, respectively.