When More Rollouts Add No Support: Semantic Coverage Collapse in Agentic Test-Time Search
Abstract
Agentic inference systems can fail while appearing active: they spend budget producing many traces, yet those traces collapse onto the same answer class, proof route, mechanism, evidence pattern, or fix strategy. We call this failure mode semantic coverage collapse and diagnose it with trace/support gaps, duplicate-token mass, residual support curves, and search–selector separation. To make the diagnostic actionable, we formalize semantic coverage, a portfolio objective that credits completed artifacts only for residual task-relevant support not already supplied by the prompt, context, or current portfolio; for fixed evaluator-induced coverage profiles, the value is normalized, monotone, and submodular. We lift this objective to bounded completion-level rollout actions and show that additive trace scoring overcounts support by the expected excess multiplicity of duplicate residual hits. These results motivate Semantic-Coverage Portfolio Search (SCPS), a receding-horizon intervention that allocates compute using predicted residual support, predicted overlap, and cost. On a 256-question held-out MMLU-Pro exact-answer evaluation with Qwen3.5-9B, SCPS reaches 91.0% portfolio Pass@16 versus 75.0% and 74.2% for tree-prefix and semantic-pruning baselines while using 31.6% of their realized tokens, but selector conversion remains a separate failure: common-selector and majority accuracies are 41.8% and 65.2%.