HypCL: Adapting CLIP in Hyperbolic Space for Continual Learning

Recently, vision-language models (e.g., CLIP) have been increasingly adopted for continual learning to mitigate catastrophic forgetting. However, existing CLIP-based methods typically freeze the backbone to preserve pre-trained knowledge, which limits the model's ability to learn discriminative features for downstream tasks. In this paper, we introduce HypCL, a parameter-efficient framework that continually adapts CLIP in hyperbolic space for continual learning. Our key insight is that the exponentially expanding capacity of hyperbolic geometry naturally accommodates the growing class space and promotes stronger inter-class separation. Specifically, HypCL attaches task-specific adapters and composes their updates sequentially in the Poincaré ball. To exploit the enhanced feature separability of hyperbolic geometry, HypCL maintains visual prototypes computed from the adapted features, which serve as stable anchors for calibrating predictions at inference. Extensive experiments on standard class-incremental benchmarks demonstrate that HypCL consistently outperforms existing CLIP-based continual learning methods.

Modern image recognition models can recognize many categories after large scale pre training, but they struggle when new categories arrive over time. Updating the model may help it learn new classes, but can also make it forget earlier ones. We propose HypCL, which gives the model a safer way to update itself as it learns new categories. Instead of simply piling new changes on top of old ones, HypCL organizes these changes in a way that leaves more room for new categories and keeps earlier knowledge easier to preserve. Experiments on multiple class incremental learning benchmarks show that HypCL improves learning of new classes while reducing forgetting. It does this without storing old training images, making it useful when storage or privacy is limited.