MOC: Multi-Order Communication in LLM-based Multi-Agent Systems

Despite the remarkable progress of Large Language Model (LLM) based Multi-Agent Systems, most research focuses on optimizing coordination topology while largely underexploring the equally critical problem: how to transmit and optimize messages among agents effectively? Current communication schemes typically rely on the direct concatenation of first-order neighbor responses, which induces a restricted evidence receptive field and leads to the dilution of crucial insights over multi-hop paths. To address these limitations, we propose the Multi-Order Communication (MOC) scheme, which reconstructs the inter-agent communication to capture multi-hop dependencies and incorporates a structural message consolidation strategy to ensure efficiency. Specifically, we formalize the communication mechanism to construct a structured multi-order evidence stream, and subsequently design a Semantic-Topological Merging algorithm to optimize semantic fidelity within token constraints. Extensive experiments across six diverse datasets and LLM backbones of varying parameter scales demonstrate that MOC consistently improves task performance and reduces communication costs.

Large language models are increasingly being organized into teams, where different agents discuss a problem before producing an answer. However, these agents often communicate in a limited way: each agent usually receives only the messages from its direct predecessors, so useful evidence from earlier agents can be lost, repeated, or distorted as it passes through the team. We propose Multi-Order Communication (MOC), a method that lets an agent see relevant messages from multiple steps upstream, not just from its immediate neighbors. MOC organizes these messages according to the team’s communication structure, so the agent can compare nearby and distant evidence in a clear order. To keep the input from becoming too long, MOC also merges messages that contain similar information while preserving their important content. This allows multi-agent language model systems to use richer evidence without greatly increasing communication cost. Across reasoning, math, and programming benchmarks, MOC improves performance over standard multi-agent communication and can reduce redundant input in larger agent teams. This work shows that improving how agents exchange information is as important as deciding how they are connected.