AURA: Visually Interpretable Affective Understanding via Robust Archetypes

Interpretable methods such as vision--language models (VLMs) have advanced affect analysis by aligning images with textual descriptions. However, relying on text as an intermediate proxy faces critical limitations: linguistic templates are inherently discrete, making them fundamentally incompatible with continuous valence--arousal regression, while also acting as a bottleneck for fine-grained visual nuances. Cognitive psychology suggests that human affective perception is not mediated by linguistic translation, but is grounded in perceptual resemblance to internalized visual archetypes. Motivated by this, we propose AURA, an archetype-based framework that replaces brittle linguistic proxies with a self-organizing archetype manifold. By adaptively allocating representational density according to affective complexity, AURA enables accurate continuous regression and reshapes affective taxonomies by decomposing labels into interpretable, geometrically coherent visual primitives. This paradigm offers a transparent, visually grounded decision trail and achieves state-of-the-art results across discrete and continuous tasks.

This paper studies how computers can better understand human facial expressions in a way that is accurate and easier to explain. Many recent systems use text descriptions, such as prompts or emotion words, to guide facial affect recognition. However, human expressions are often subtle, mixed, and continuous. A short text label may not capture the difference between a weak smile, a tense smile, or a surprised smile. We propose AURA, a method that learns visual reference patterns directly from face images. Instead of asking the model to match a face to a written description, AURA compares the face with learned visual examples, called archetypes. These archetypes represent recurring facial patterns, such as different kinds of smiles, surprise, sadness, or small facial movements. The model can also learn more archetypes for complex expressions and fewer for simpler ones. This makes the model both effective and more transparent. It can explain which visual pattern influenced a prediction, help identify ambiguous or possibly noisy labels in datasets, and support different affective tasks, including expression recognition, action unit detection, and continuous emotion estimation. Experiments on several benchmarks show that AURA improves performance while keeping the prediction process lightweight and interpretable.