Probabilistic Salient Object Ranking

Salient Object Ranking (SOR) aims to study how humans visually explore complex scenes by predicting an ordered sequence of objects that attracts our attention. Existing SOR approaches typically model this ranking deterministically, assuming a single, fixed ranking sequence of attention. However, such deterministic SOR fails to capture the true nature of human attention. We observe that human attention shifts exhibit variability and stochasticity, i.e., the next object of fixation is not a definitive choice but rather a probability distribution. Yet, existing SOR methods and evaluation metrics do not account for this inherent randomness. To address this fundamental problem, we first propose ProbSOR, a novel Probabilistic Salient Object Ranking framework built upon a vision-language model (VLM) backbone. By incorporating Group Relative Policy Optimization (GRPO), ProbSOR explicitly learns the uncertainty of attention shifts. We then propose a new metric tailored for ProbSOR, as existing SOR metrics only support deterministic rankings. We further construct a ProbSOR dataset comprising 15,000 probabilistic SOR samples, to support both model training and evaluation. Extensive experiments show that ProbSOR achieves strong performance in salient object ranking under both our proposed and traditional benchmarks.

When people look at a complex image, they do not always notice the same objects in the same order. However, most existing methods for salient object ranking ask a computer to predict one fixed “correct” ordering of the important objects, which can unfairly penalize other reasonable viewing patterns. We propose ProbSOR, a method that treats human attention as a distribution of possible paths rather than a single sequence. Using human eye-fixation data, we build a tree of likely attention shifts and train a vision-language model to predict object rankings that better match these patterns. We also introduce a benchmark and evaluation scores that give credit to predictions that follow any plausible human attention path, not only the most common one. This makes the evaluation closer to how people actually explore visual scenes. In experiments, ProbSOR better captures diverse human-like attention shifts while remaining competitive at locating the relevant objects. This work can help vision systems reason about complex images in a more flexible and human-centered way.