Agent Primitives: Reuseable Latent Building Blocks for Multi-Agent Systems

While existing multi-agent systems (MAS) can handle complex problems by enabling collaboration among multiple agents, they are often highly task-specific, relying on manually crafted agent roles and interaction prompts, which leads to increased architectural complexity and limited reusability across tasks. Moreover, most MAS communicate primarily through natural language, making them vulnerable to error accumulation and instability in long-context, multi-stage interactions within internal agent histories. In this work, we propose \textbf{Agent Primitives}, a set of reusable latent building blocks for LLM-based MAS. Inspired by neural network design, where complex models are built from reusable components, we observe that many existing MAS architectures can be decomposed into a small number of recurring internal computation patterns. Based on this observation, we instantiate three primitives (Review, Voting and Selection, and Planning and Execution), all communicating via key–value (KV) cache to mitigate information degradation across multi-stage interactions. To enable automatic system construction, an Organizer agent automatically selects and composes primitives for each query, guided by a lightweight knowledge pool of previously successful configurations, forming a primitive-based MAS. Experiments show that primitives-based MAS improve average accuracy by 12.0–16.5\% over single-agent baselines, reduce token usage and inference latency by approximately 3$\times$–4$\times$ compared to text-based MAS, while incurring only 1.3$\times$–1.6$\times$ overhead relative to single-agent inference and providing more stable performance across model backbones.

AI systems are increasingly built from teams of AI agents that collaborate to solve complex problems, much like how human teams divide up work. However, these multi-agent systems are hard to design: engineers must manually craft specialized roles and communication protocols for every new task, and the systems often break down in long conversations because agents pass information as plain text that gets garbled or ignored over time. We introduce Agent Primitives, a small set of reusable building blocks for assembling AI agent teams, inspired by how modern neural networks are built from standard components like attention layers. We identified three fundamental collaboration patterns that appear repeatedly across existing systems: Review (one agent critiques another's work), Voting and Selection (multiple agents propose solutions and the best one is chosen), and Planning and Execution (one agent plans, others carry it out). Crucially, these primitives pass information through the model's internal memory rather than as text, making communication more robust and efficient. An automated coordinator assembles the right combination of primitives for each task. Across eight diverse benchmarks covering math, coding, and question answering, our approach improves accuracy by 12–16\% over single-agent baselines while using 3$\times$–4$\times$ fewer tokens than conventional multi-agent systems, making collaborative AI both more capable and more practical to deploy.