Weaving in the Clouds: Achieving Synergistic Collaboration among LLM Agents via Federated Learning

Multi-Agent Systems (MAS) powered by Large Language Models (LLMs) have become a promising paradigm for solving workflow-structured tasks through expert collaboration. However, the data required for such collaboration are often distributed across organizations and cannot be centrally pooled due to privacy, intellectual property, and compliance constraints. Federated Learning preserves data locality, but existing federated paradigms typically treat clients as independent and fail to model workflow dependencies that are crucial for coherent multi-stage collaboration. We introduce FedWave, a federated and workflow-aware framework that enables LLM-based experts to solve sequential tasks under data-locality constraints. FedWave combines a Value Chain Layer to model inter-stage dependencies, federated LoRA adaptation to preserve role-specific expertise, a shared Mixture-of-Experts (MoE) router for input-conditioned expert fusion, and a Direct Preference Optimization (DPO) stage based on router-induced preferences. Experiments across workflow datasets and LLM backbones show that FedWave outperforms federated baselines and remains competitive with centralized multi-agent systems while preserving data locality.

In many business workflows, experts at different stages need to work together to solve a complex task. However, because of privacy, legal, or business constraints, they often cannot share their data with one another. This creates a challenge for collaborative AI: how can specialized language-model agents learn from local data while still working together as a team? Our work introduces FedWave, a method that enables these agents to “weave in the clouds.” Each agent learns from its own private data locally, while a cloud-based collaboration mechanism coordinates their knowledge across the workflow. FedWave helps the system understand how different stages depend on one another, decide when each expert should contribute, and produce more coherent joint answers. By allowing organizations to keep sensitive data local while still building collaborative AI systems, FedWave supports cross-organizational cooperation in business workflows.