MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems

Multi-Agent Systems (MAS) built on Large Language Models (LLMs) often exhibit high variance in their reasoning trajectories. Process verification, which evaluates intermediate steps in trajectories, has shown promise in general reasoning settings, and has been suggested as a potential tool for guiding coordination of MAS; however, its actual effectiveness in MAS remains unclear. To fill this gap, we present MAS-ProVe, a systematic empirical study of process verification for multi-agent systems (MAS). Our study spans three verification paradigms (LLM-as-a-Judge, reward models, and process reward models), evaluated across two levels of verification granularity (agent-level and iteration-level). We further examine five representative verifiers and four context management strategies, and conduct experiments over six diverse MAS frameworks on multiple reasoning benchmarks. We find that process-level verification does not consistently improve performance and frequently exhibits high variance, highlighting the difficulty of reliably evaluating partial multi-agent trajectories. Among the methods studied, LLM-as-a-Judge generally outperforms reward-based approaches, with trained judges surpassing general-purpose LLMs. We further observe a small performance gap between LLMs acting as judges and as single agents, and identify a context-length-performance trade-off in verification. Overall, our results suggest that effective and robust process verification for MAS remains an open challenge, requiring further advances beyond current paradigms.

Large Language Models (LLMs) are increasingly being deployed as Multi-Agent Systems (MAS), where multiple LLM-based agents collaborate to solve complex problems. However, the step-by-step reasoning within these collaborations can be highly chaotic. Evaluating a single agent's intermediate work (a technique known as process verification) is a proven way to keep it on track, but the effectiveness of this automated oversight remains unknown when multiple agents coordinate. To find out, we conducted a systematic study to test various methods for evaluating the partial, in-progress work of these systems. We experimented with different types of automated judges, varying levels of scrutiny, and multiple MAS architectures. We also released our testing framework publicly to allow researchers to rigorously evaluate their own multi-agent pipelines. Surprisingly, we found that process verification is not a magic bullet. It does not consistently improve the final performance of the MAS and, without carefully optimized settings, produces highly unstable results. Our results reveal that effectively steering collaborative systems remains a major unsolved challenge, highlighting that evaluation methods built for individual agents simply do not seamlessly transfer to team-based systems.