LLM4Cov: Execution-Aware Agentic Learning for High-coverage Testbench Generation

Execution-aware LLM agents offer a promising paradigm for learning from tool feedback, but such feedback can be expensive and slow to obtain, making online reinforcement learning (RL) less practical in certain scenarios. High-coverage hardware verification exemplifies this challenge due to its reliance on industrial simulators and non-differentiable execution signals. We propose LLM4Cov, an offline agent-learning framework that models verification as single-step state transitions guided by deterministic evaluators. Building on this formulation, we introduce execution-validated data curation, policy-aware agentic data synthesis, and worst-state-prioritized sampling to enable scalable learning under execution constraints. We further curate a reality-aligned benchmark adapted from an existing verification suite through a revised evaluation protocol. Using the proposed pipeline, a compact 4B-parameter model achieves 69.2\% pass rate and 90.4\% average coverage in CVDP-ECov under agentic evaluation, outperforming its teacher by 5.3\% and 10.5\%, demonstrating competitive performance against models an order of magnitude larger.

Before a computer chip is manufactured, engineers must check its design exhaustively, because a flaw baked into silicon can't be patched afterward the way software can. This checking — writing test programs that exercise every part of the design — accounts for most of the effort in chip development. AI language models could help, but teaching them normally means letting them learn by trial and error, which requires running slow, expensive hardware simulators thousands of times. We built LLM4Cov, a method that trains a chip-testing AI without that costly trial-and-error loop. Rather than having the model carry its full history forward, we let it focus only on its latest attempt and the feedback on it. We then concentrate training on its weakest attempts — teaching it to rescue tests that exercise too little of the design — and we train it in stages that keep pace as it improves, first imitating a stronger "teacher" model, then refining its own work. A small model trained this way outperformed its much larger teacher and matched models fifty to a hundred times its size, showing that careful, targeted training can beat sheer scale and make automated chip verification far cheaper.