Timezone: »

 
Prompt-based Generative Replay: A Text-to-Image Approach for Continual Learning in Medical Settings
Yewon Byun · Saurabh Garg · Sanket Vaibhav Mehta · Jayashree Kalpathy-Cramer · Praveer Singh · Bryan Wilder · Zachary Lipton

Sat Jul 29 05:10 PM -- 05:20 PM (PDT) @
in Machine Learning for Multimodal Healthcare Data »

Episodic replay methods, which store and replay past data, have been effective in handling distribution shifts in continual learning. However, due to regulatory and privacy concerns for data sharing, their applicability can be limited. In this work, we introduce two novel healthcare benchmarks for domain incremental continual learning: diabetic retinopathy severity classification and dermoscopy skin lesion detection, and highlight issues of poor forward and backward transferability in simple baselines. To overcome these challenges, we propose a novel method called prompt-based generative replay. By leveraging a text-to-image diffusion model for synthetic data generation, our approach effectively preserves previously learned knowledge while adapting to new data distributions. Our experiments demonstratethat our prompt-based generative replay significantly outperforms competitive baselines, resulting in an average increase of up to 5 points in average AUC for the skin lesions benchmark and up to 2 points for the diabetic retinopathy benchmark.

Keywords: Medical Imaging · Benchmarking, domain shifts, and generalization

Author Information

Yewon Byun (Carnegie Mellon University)
Saurabh Garg (Carnegie Mellon University)
Sanket Vaibhav Mehta (Carnegie Mellon University)
Jayashree Kalpathy-Cramer (University of Colorado School of Medicine)
Praveer Singh (University of Colorado School of Medicine)
Bryan Wilder (Carnegie Mellon University)
Zachary Lipton (0)

More from the Same Authors