Beyond Single Embedding: Modeling User Preferences as Distribution in Federated Recommendation

Most federated recommender systems represent each user with a single embedding learned from local interaction data, implicitly assuming that user preferences are fixed and precisely identifiable. In federated settings, however, each client observes only a limited and fragmentary view of user behavior, rendering such point estimates inherently brittle. To address this mismatch, we model user preferences as distributions rather than points, allowing multiple compatible preference representations to coexist. Rather than collapsing evidence into a single embedding, our approach preserves uncertainty and diversity in user representations, providing a richer basis for preference modeling. We instantiate this idea with a diffusion-based generative framework that produces diverse user embeddings and derives recommendation scores by aggregating predictions across them. This distributional formulation yields more stable ranking behavior and improved robustness under ambiguous feedback. Extensive experiments on federated recommendation benchmark datasets demonstrate consistent and significant improvements over baselines. Our code is available.

This research was prompted by a basic problem: federated recommenders try to capture a user's taste with a single profile learned from limited local data, but when each device only sees a small slice of behavior, that profile is often confused and unstable. We tackled this by treating user preferences as a range of possibilities rather than one fixed guess, using a generative model to create multiple plausible user profiles and blending their predictions. This matters because it makes recommendations more reliable when data is sparse and user signals are unclear.