AutoBaxBuilder: Bootstrapping Code Security Benchmarking

As large language models (LLMs) see wide adoption in software engineering, the reliable assessment of the correctness and security of LLM-generated code is crucial. Notably, prior work showed that LLMs are prone to generating code with security vulnerabilities, highlighting that security is often overlooked. These insights were enabled by specialized benchmarks crafted by security experts through significant manual effort. However, benchmarks (i) inevitably end up contaminating training data, (ii) must extend to new tasks to provide a more complete picture, and (iii) must increase in difficulty to challenge more capable LLMs. In this work, we address these challenges and present AutoBaxBuilder, an automated pipeline that generates code security benchmarking tasks from scratch. It leverages the code-understanding capabilities of LLMs combined with robust reliability checks to construct functional tests and end-to-end security-probing exploits. The quality of the pipeline is quantitatively confirmed by aligning its predictions with an expert-written baseline and qualitatively validated through manual soundness verification. We use AutoBaxBuilder to construct a new benchmark and release it to the public as AutoBaxBench, together with a thorough evaluation on contemporary LLMs. AutoBaxBuilder generates new tasks in under 2 hours, for less than USD 4. Including a manual verification, this reduces the required human effort for benchmark construction by a factor of 12.

Using large language models (LLMs) to write code has become a widespread practice. Even if LLM-generated code may work as intended to a benign user, it may be susceptible to attacks by a malicious entity. Web applications are exposed to the open internet, and as such exposed to attackers. To measure the safety of using LLMs to code such applications, we need benchmarks: collections of programming tasks that check whether generated code is both useful and safe. Building benchmarks by hand takes security expertise and many hours, and old benchmarks can become less useful once their tasks appear in model training data. We present AutoBaxBuilder, a system that cheaply and fully automatically creates novel benchmark tasks from scratch. It defines a web application scenario, generates tests for the expected benign behavior, and develops attacks that should succeed on vulnerable code but fail on patched code. We observe trends that are consistent with expert-written tests: the same AI systems tend to look strong or weak, and our generated attacks often catch additional security problems. We publicly release AutoBaxBench, a benchmark created with AutoBaxBuilder that complements existing expert-written tasks. Our evaluation shows that many current AI models can produce code that passes normal tests but still contains exploitable security bugs.