AgentTailor: A Semantic-Aware LLM-Based Multi-Agent System with Actor-Critic Structure

Large Language Model (LLM)-based multi-agent systems often suffer from high communication cost due to redundant interactions, as existing methods optimize communication structures without explicitly measuring whether exchanged messages contribute to the final decision. To better utilize the semantic information in the execution stage to further optimize the structure of multi-agent systems and reduce token costs, we propose AgentTailor, a cost-aware framework that evaluates the semantic contribution of communication edges via an edge judgment mechanism, and employs an Edge Prediction Network (EPN) to estimate edge utilities through virtual execution without invoking LLMs. Experiments show that AgentTailor achieves the best average accuracy (91.36\%) on six datasets of diverse fields, while reducing total tokens by 21.2\%--61.6\%. Our work demonstrates that explicitly modeling semantic edge contribution is crucial for scalable and efficient multi-agent systems, providing a principled approach to communication optimization that goes beyond structural heuristics. The code is available at https://github.com/Pt3Y/AgentTailor.

LLM-based multi-agent systems waste tokens and latency on many messages that barely change the final answer. Even when a rollout succeeds, RL-style training often assigns uniform rewards to every participating edge, although their contributions to that answer differ. AgentTailor uses edge-level credit: each directed edge is scored by semantic alignment with the round’s aggregated decision, with per-source relative normalization so late paths are not favored merely for lying closer to the final text. A compact Edge Prediction Network imitates judge-derived utilities, enabling virtual graph refits per problem without extra LLM calls. On six diverse benchmarks it reaches 91.36% average accuracy and cuts total tokens by 21.2%–61.6% versus AgentDropout under matched settings, improving the quality–cost trade-off over purely structural heuristics.