Attend to Anything: Foundation Model for Unified Human Attention Modeling

Existing human attention (saliency) modeling methods persist as highly fragmented across modalities, scenes, and task formulations. Consequently, even with increasing model capacity and data scale, current models predominantly remain scene-dependent and task-specific, failing to practically generalize in real-world applications. To address the fundamental limitations, we present the Attend to Anything Model (AAM), a multi-modal foundation model that unifies attention modeling across various image, video, and audio-visual tasks and scenes. AAM reformulates attention as a cognitive entailment relationship organized in a general-to-specific hierarchy, implemented through language prompts with hierarchical embeddings in hyperbolic space. Furthermore, to unify static image and dynamic video attention, we adopt a fluid-dynamics perspective, formulating video-frame attention as a diffusive temporal evolution governed by the Fokker--Planck equation. Extensive experiments on 16 benchmarks demonstrate that AAM consistently outperforms state-of-the-art methods by an average of 6\% across various scenarios, while achieving approximately a 4$\times$ speedup in video inference. Overall, these results demonstrate that AAM provides a principled foundation for future research on attention and saliency-related tasks. The dataset and code will be available at \url{https://github.com/wz-zhao/Attend-to-Anything}.

Human attention naturally focuses on the most important regions when people view images and videos or perceive sounds. However, existing attention and saliency models are often limited to specific scenarios or task settings, such as static images, particular types of videos, or individual audio-visual contexts. As a result, they struggle to generalize flexibly in real-world applications. We propose the Attend to Anything Model (AAM), a unified model for understanding attention across images, videos, and audio-visual scenes. AAM formulates attention as a cognitive entailment relationship, enabling the model to reason not only about what is salient, but also about hierarchical task relationships from general to specific through language prompts. To bridge static image attention and dynamic video attention, we further draw inspiration from fluid dynamics and model the temporal evolution of attention in videos as a diffusion process over time. Across 16 public benchmarks, AAM consistently outperforms existing state-of-the-art methods by an average of approximately 6% on diverse attention and saliency tasks, while achieving about a 4× speedup in video inference. These results suggest that AAM can serve as a more general and efficient foundation model for attention modeling, supporting future research on predicting where humans attend across image, video, and audio-visual tasks.