Unsupervised Process-Aware Coreset Selection for In-Context Learning

We address the challenge of unsupervised coreset selection for few-shot in-context learning (ICL). The goal is to select a small subset of examples under a fixed annotation budget to yield effective prompts for large language models. Existing geometry-based methods often yield coresets that suffer from a skewed distribution, due to the oversampling of peripheral examples and high local redundancy. To address these issues, we propose a process-aware framework for coreset selection. It jointly optimizes the diversity and representativeness of selected samples via an adaptive submodular objective. It ensures representativeness by selecting samples based on local density awareness, while promoting diversity by imposing a redundancy penalty relative to the evolving selected set. Thus, it performs progress-aware balancing of representativeness and diversity based on the selection context. Extensive experiments on 7 NLP datasets demonstrate that our method consistently outperforms state-of-the-art coreset selection methods in downstream ICL performance. Further analysis validates that our approach better balances diversity and representativeness in the selection process, while retaining the theoretical guarantees of adaptive submodular optimization.

Large language models (LLMs) can perform tasks by looking at just a few examples, a capability called in-context learning. But which examples work better? Selecting a small, effective set from a large pool of unlabeled texts is challenging. Our research aims to avoid picking too many similar examples or rare edge cases under a limited annotation budget, thus achieving better LLM performance. We propose a new method that picks examples in a principled, adaptive way. It balances two goals: selecting core examples that represent many others (representativeness) while avoiding redundancy (diversity). It works step by step—after each selection, it automatically re-evaluates the remaining candidates to prevent picking similar ones again. It also adjusts its strategy based on the annotation budget: when only a few examples can be labeled, it prioritizes the most typical ones; when the budget is larger, it shifts toward diversity. This makes LLMs more sample-efficient, reducing the need for massive labeled datasets. In practice, it helps developers build better text analysis tools with far less manual annotation work.