HEDP: A Hybrid Energy-Distance Prompt-based Framework for Domain Incremental Learning

Domain Incremental Learning is a critical scenario that requires models to continuously adapt to new data domains without retraining. However, domain shifts often cause severe performance degradation. To address this, we propose Hybrid Energy-Distance Prompt, a domain-incremental framework inspired by Helmholtz free energy. HEDP introduces an energy regularization loss to enhance the separability of domain representations and a hybrid energy-distance weighted mechanism that fuses energy-based and distance-based cues to improve domain selection and generalization. Experiments on multiple benchmarks, including CORe50, show that HEDP achieves superior performance on unseen domains with a 2.57\% accuracy gain, effectively mitigating catastrophic forgetting and enhancing open-world adaptability. Our code is available at \url{https://github.com/dannis97500/HEDP}.

Machine learning systems often struggle when the world changes after they are trained. For example, an image model that works well in one environment may become unreliable when images come from a new style, device, or condition. This paper studies how to help such models keep learning across different domains without retraining the whole model or storing old training data. We propose HEDP, a method that gives each learned domain a small prompt and then decides how useful each prompt is for a new test image. The key idea is to combine two signals: how close the image looks to previous domains, and how well each domain-specific prompt “fits” the image, measured using an energy-inspired score. We also train the prompts so that these energy scores are easier to compare across domains, reducing confusion when domains overlap. Across three image-recognition benchmarks, HEDP better preserves performance on known domains and improves accuracy on unseen domains. This makes continual AI systems more practical for changing real-world settings, such as autonomous driving, robotics, or visual inspection, where new conditions appear over time.