When Generalized Zero-Shot Learning Meets PU Learning: A Plug-and-Play Framework for Seen-Class Bias Mitigation
Abstract
Generalized Zero-Shot Learning (GZSL) suffers from severe seen-class bias, a challenge stemming from the label incompleteness inherent in the mixed seen–unseen test set. To address this, we propose PUFE, a unified plug-and-play post-processing framework that recasts GZSL inference as a Positive-Unlabeled (PU) learning task by treating seen categories as positive and mixed test data as unlabeled. PUFE constructs a PU classifier in the semantic space, jointly estimating the seen-class posterior and labeling propensity via Maximum Likelihood Estimation (MLE) within a dual-head network. Furthermore, we introduce an adaptive prototype calibration strategy that employs high-confidence pseudo-instances—identified by the PU classifier—to explicitly align semantic prototypes with the underlying test distribution. Extensive experiments demonstrate that PUFE mitigates bias and raises the performance of diverse GZSL baselines across multiple architectural paradigms, yielding gains of up to 11.2 percentage points in the harmonic mean.
Lay Summary
When an AI system is trained to recognize objects—say, different species of birds—it only sees labeled examples from a limited set of categories. At test time, however, it is asked to classify images from both those familiar categories and entirely new ones it has never encountered. The trouble is that the model is strongly biased toward what it already knows: it tends to force unfamiliar images into familiar categories, even when they clearly do not belong there. This is a well-known weakness of so-called generalized zero-shot learning systems, and it remains a stubborn obstacle to deploying them in the real world. We addressed this by borrowing an idea from a different area of machine learning called positive-unlabeled (PU) learning. The key insight is simple: during inference, we know which categories the model was trained on, but we do not know which test images belong to new categories. This is exactly the situation PU learning was designed for—you have some confirmed "positive" examples and a large pool of unlabeled ones. We built a lightweight module called PUFE that sits on top of any existing system and uses this structure to figure out, at test time, which images are likely from new categories and which are not. It then uses those estimates to nudge the model's internal reference points—the templates it uses to match images to categories—closer to where they should actually be. PUFE requires no retraining of the underlying model and can be plugged into a wide range of existing systems. Across standard benchmarks, it consistently improved recognition performance, in some cases by more than 11 percentage points. The results suggest that explicitly accounting for the "who labeled what" structure of the test data is a practical and principled way to make zero-shot recognition systems less biased and more reliable.