When Attributes Disagree: Gradient Conflict in Image Aesthetic Assessment

Image Aesthetic Assessment (IAA) predicts an image’s overall aesthetic score, yet aesthetic is influenced by multiple attributes whose relative importance varies with image content and usage scenarios. Under end-to-end training with only overall-score supervision, attribute signals are blended, which can cause gradient conflict across samples dominated by different attributes, resulting in gradient cancellation and persistent systematic bias. To address these issues, we propose AGREE (Attribute-guided Gradient Routing for Establishing Agreement), which learns attribute-specific subspaces and performs gradient routing based on sample-wise attribute sensitivity estimated via perturbation analysis. AGREE further reduces feature coupling across attributes with semantic anchors and improves robustness via error-aware reweighting. Experiments on AVA, LAPIS, AADB, TAD66K, and PARA show consistent improvements over diverse IAA baseline models, and AGREE is plug-and-play for existing end-to-end IAA methods without modifying their original architectures. To our knowledge, this work is among the early efforts in IAA to systematically study gradient conflict and provide an effective solution. The code is available at https://dahat364.github.io/AGREE/.

People often judge the beauty of an image by considering many visual factors, such as brightness, color, sharpness, and contrast. However, most current image aesthetic assessment models are trained to predict only one overall beauty score. This can make the model mix different visual factors together, and the training process may favor some factors while neglecting others. As a result, different models can make similar mistakes on the same difficult images. In this work, we study why these shared mistakes happen and propose AGREE, a plug-in method that helps existing models handle different visual factors more separately. AGREE creates separate learning paths for several image attributes, estimates which attributes matter more for each image, and gives more training attention to images that are repeatedly hard to predict. Experiments on five image aesthetic datasets and six existing models show that AGREE consistently improves prediction accuracy, especially on difficult images where previous models often fail. This suggests that better understanding attribute-level differences can make aesthetic assessment models more reliable and fairer across diverse images.