The Token Tax : Measuring the Diminishing Returns of Test-Time Compute in Agentic Pipelines
Abstract
The assumption that increased test-time reasoning improves performance is driving the adoption of autonomous LLM agents in data science pipelines. Using the Living Agentic System (LAS), we show that this assumption fails in ML workflows due to a ``Token Tax,'' where larger reasoning budgets sharply increase token costs with marginal utility gains. Across DeepSeek-7B, Mistral-7B, and Llama-3.1:8B, we observe a stable efficiency frontier in which low planning budgets achieve correct pipeline behavior, while additional reasoning results in planning inflation, increasing cost by over 110\% with negligible quality improvement. We further demonstrate that intent-aware governance eliminates leakage-related false positives, reducing them from 74.5\% to 0\%, providing a practical foundation for efficient and reliable agentic data science systems.