$\tau$-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains

Full-duplex voice agents—systems that listen and speak simultaneously—are rapidly moving from research to production. However, existing evaluations address conversational dynamics and task completion in isolation. We introduce τ-Voice, a benchmark for evaluating voice agents on grounded tasks with real-world complexity: agents must navigate complex multi-turn conversations, adhere to domain policies, and interact with the environment. The framework extends τ²-bench into a novel voice agent benchmark combining verifiable completion of complex grounded tasks, full-duplex interaction, and realistic audio—enabling direct comparison between voice and text performance. A controllable and realistic voice user simulator provides diverse accents, realistic audio environments, and rich turn-taking dynamics; by decoupling simulation from wall-clock time, the user simulator can use the most capable LLM without real-time constraints. We evaluate task completion (pass@1) and voice interaction quality across 278 tasks: while GPT-5 (reasoning) achieves 85%, voice agents reach only 31–51% under clean conditions and 26–38% under realistic conditions with noise and diverse accents—retaining only 30–45% of text capability; qualitative analysis confirms 79–90% of failures stem from agent behavior, suggesting that observed failures primarily reflect agent behavior under our evaluation setup. τ-Voice provides a reproducible testbed for measuring progress toward voice agents that are natural, conversational, and reliable.

AI voice assistants — the kind that listen and speak in real time — are starting to handle customer-service calls for returns, account changes, and billing issues. But how reliable are they really, once you put them through the noisy, messy reality of a phone call? We built τ-Voice, an open benchmark with 278 customer-service tasks across retail, airline, and telecom, each one solved against a real database so we can check whether the agent actually got the job done. We tested the leading voice AI systems from OpenAI, Google, and xAI under both ideal and realistic audio conditions, including background noise, regional accents, and the natural give-and-take of a real conversation. Even on clean audio, the best voice agent completed only 31–51% of tasks; under realistic conditions the rate dropped to 26–38%. A text-only AI given the same tasks solves 85%, so the gap is large and largely attributable to the voice modality itself. Most failures aren't deep reasoning failures — they're basic things people do without thinking: spelling a name correctly when authenticating a customer, asking for confirmation before making an irreversible change, or keeping track of a multi-part request. τ-Voice is open so the community can measure progress as voice AI continues to improve.