Beyond Correctness: Distance-Based Social Dynamics of Multi-Agent Debate

Multi-agent debate (MAD) systems are often evaluated using binary correctness or peer agreement, yet such evaluations obscure how individual agents revise their answers during social interaction. We study the microscopic dynamics of answer revision in large language models (LLMs) using ConceptARC, a 2D grid-reasoning benchmark that admits quantitative distance measures between candidate solutions. By exposing a target model to controlled configurations of peer answers, we analyze how the likelihood and direction of revision depend on both social context and the distance between answers and the ground truth. We find that agents are more likely to revise when their answers are farther from the correct solution, and that revisions of incorrect answers exhibit a systematic contraction toward the ground truth, even when the final answer remains incorrect. Conversely, correct answers can be overturned by social pressure, particularly when wrong peers are near-correct. Together, these results show that multi-agent interaction induces structured, distance-aware movements in solution space that are invisible under binary correctness, clarifying when social reasoning leads to improvement, stability, or gradual regression in solution quality.

When AI language models discuss a problem together, they often revise their answers after seeing peers' responses. Researchers usually study this by counting how often the AIs end up right or wrong, but that's like watching a chess match and only recording who won: missing every move in between. We used a visual puzzle benchmark in which answers are small grids, allowing us to measure not just whether an answer is wrong but also how wrong it is. By placing an AI agent among carefully chosen peers, we tracked how its answers shifted across two leading models. The AIs behaved in surprisingly human-like ways: more willing to change their mind when their answers were more incorrect, while even correct AIs can be steered by near-correct peers, mirroring well-known patterns of human social conformity and confidence under uncertainty. As AI agents increasingly work in groups, designing effective AI societies and organizations cannot rely on right-or-wrong outcomes alone. Our distance-based view reveals hidden learning signals and vulnerabilities that shape whether collective reasoning improves, stagnates, or quietly regresses.