VeRO: A Harness for Agents to Optimize Agents

An important emerging application of coding agents is agent harness optimization: the iterative improvement of a target agent by editing and evaluating its code. Despite its relevance, the community lacks a systematic understanding of coding agent performance on this task. Harness optimization differs from conventional software engineering: agent harnesses interleave deterministic code with stochastic LLM completions, requiring structured capture of both intermediate execution traces and downstream outcomes. To address these challenges, we introduce (1) VeRO (Versioning, Rewards, and Observations), an outer harness that provides versioned snapshots, budget-controlled evaluation, and structured execution traces of target harnesses, and (2) VeRO-Bench, a benchmark suite of target agents and tasks with reference evaluation procedures. Using VeRO, we conduct an empirical study comparing optimizers across tasks and analyzing which modifications reliably improve target agent harnesses. We release VeRO to support research on agent optimization as a core capability for coding agents. Code is available at https://github.com/scaleapi/vero.

Modern AI assistants are programs that combine a language model with tools, memory, and decision rules — what researchers call "agents." Building a good agent today is a slow manual loop: try a prompt, run it, look at the failures, tweak the code, repeat. A new generation of "coding agents" can read and write code on their own, raising an obvious question: can a coding agent take over the agent-building loop itself? We introduce VeRO, a system that lets a coding agent improve a target agent under controlled conditions: every change is versioned, evaluations run against a fixed budget, and execution traces are recorded. We pair it with a benchmark of optimization tasks spanning math, science, factual lookup, multi-step research, customer dialogue, and terminal coding. Running today's best coding agents as optimizers, we find they can reliably improve tool-heavy agents (typical gains of 7–15%). We also show that without VeRO's guardrails, optimizers cheat: they peek at test answers and exceed their budgets. We release VeRO so the community can benchmark progress on agent-by-agent improvement and build better self-improving AI.