Trustworthy Federated Label Distribution Learning under Annotation Quality Disparity

Label Distribution Learning (LDL) models supervision as an instance-wise probability distribution, enabling fine-grained learning under inherent ambiguity, but its success relies on high-fidelity label distributions that are costly to obtain and thus often noisy. Motivated by privacy-sensitive applications, we study Federated Label Distribution Learning (Fed-LDL), where data isolation further induces heterogeneous annotation quality across clients, making local updates unevenly reliable and breaking sample-size-based aggregation (e.g., FedAvg). To address this trust dilemma, we propose FedQual, a quality-aware Fed-LDL framework with two coupled mechanisms: (i) quality-adaptive client training guided by a global semantic anchor that calibrates low-quality clients while preserving high-quality autonomy, and (ii) reliability-aware server aggregation that reweights client contributions by effective reliable information rather than raw sample size. To enable rigorous evaluation, we construct four new Fed-LDL benchmarks (FER-LDL, FI-LDL, PIPAL-LDL, and KADID-LDL) with controlled annotation quality disparity. We further provide a theoretical guarantee showing that under heterogeneous supervision quality, client-specific calibration is strictly better than any uniform calibration. Experiments on the proposed benchmarks demonstrate the effectiveness of FedQual.

Many real-world AI systems learn from data labeled by people, but human labels are often uncertain or inconsistent. This is especially true in tasks such as judging image quality or interpreting facial emotions, where different people may reasonably give different answers. The problem becomes harder when data are kept in separate places, such as hospitals, personal devices, or institutions, because privacy rules prevent all data from being collected in one central location. In such settings, some participants may provide very reliable labels, while others may provide noisier ones. This paper studies how to train a shared AI model when different participants have labels of different quality. We propose FedQual, a method that gives more guidance to participants with less reliable labels while allowing more reliable participants to keep more influence. It also changes how updates from different participants are combined, so that a large amount of low-quality data does not dominate the final model. To support evaluation, we build four new benchmarks covering facial emotion recognition and image quality assessment. Experiments show that FedQual produces more reliable predictions under different kinds of label noise and data imbalance.