Skip to yearly menu bar Skip to main content


Attentional Meta-learners for Few-shot Polythetic Classification

Ben Day · Ramon Viñas Torné · Nikola Simidjievski · Pietro Lió

Hall E #503

Keywords: [ DL: Attention Mechanisms ] [ MISC: Transfer, Multitask and Meta-learning ]


Polythetic classifications, based on shared patterns of features that need neither be universal nor constant among members of a class, are common in the natural world and greatly outnumber monothetic classifications over a set of features. We show that threshold meta-learners, such as Prototypical Networks, require an embedding dimension that is exponential in the number of task-relevant features to emulate these functions. In contrast, attentional classifiers, such as Matching Networks, are polythetic by default and able to solve these problems with a linear embedding dimension. However, we find that in the presence of task-irrelevant features, inherent to meta-learning problems, attentional models are susceptible to misclassification. To address this challenge, we propose a self-attention feature-selection mechanism that adaptively dilutes non-discriminative features. We demonstrate the effectiveness of our approach in meta-learning Boolean functions, and synthetic and real-world few-shot learning tasks.

Chat is not available.