Compute-Aware Mixture-of-Agents: Verifier-Gated Adaptive Aggregation under a Fixed Token Budget

Mixture-of-Agents (MoA) aggregates responses from several LLMs in a fixed number of layers, spending the same compute on every query. We extend MoA with a self-consistency verifier that decides, per query, when to stop aggregating and which agents to invoke; we call the result Compute-Aware MoA (CA-MoA). On GSM8K with a heterogeneous Llama-3.2-3B + Qwen2.5-3B pool, CA-MoA reaches $85.0\%$ accuracy. That is the best score in the experiment, equal to $4$-sample self-consistency, and $5$ points above fixed-depth MoA-1 at $1.4\times$ MoA-1's tokens. We also run two stress tests where this win is absent: a homogeneous Llama-3.2-3B$\times 3$ pool where MoA-1 already saturates the agents at $92\%$, and a mixed-capability pool where adding a smaller Gemma2-2B agent corrupts aggregation. Across all three settings, CA-MoA's verifier signal still tracks correctness: queries that the gating policy lets terminate after one layer are answered correctly $89$-$100\%$ of the time, while queries that exhaust the budget are answered correctly $0$-$40\%$ of the time. The within-MoA gain over MoA-1 moves from $-8$ to $+13.3$ to $+5$ as the pool becomes more aggregation-appropriate. We pre-register a $\sim$7B similar-capability follow-up at $n=200$.