Poison with Style: A Practical Poisoning Attack on Code Large Language Models
Abstract
Code Large Language Models (CLLMs) serve as the core of modern code agents, enabling developers to automate complex software development tasks. In this paper, we present Poison-with-Style (PwS), a practical and stealthy model poisoning attack targeting CLLMs. Unlike prior attacks that assume an active adversary capable of directly embedding explicit triggers (e.g., specific words) into developers' prompts during inference, PwS leverages developers' code styles as covert triggers implicitly embedded within their prompts. PwS introduces a novel data collection method and a two-step training strategy to fine-tune CLLMs, causing them to generate vulnerable code when prompts contain trigger code styles while maintaining normal behavior on other prompts. Experimental results on Python code completion tasks show that PwS is robust against state-of-the-art defenses and achieves high attack success rates across diverse vulnerabilities, while maintaining strong performance on standard code completion benchmarks. For example, PwS-poisoned models generate CWE-20 vulnerable code in 95\% of cases when the trigger code style is used, with less than a 5\% drop in pass@1 performance on the HumanEval and MBPP benchmarks. Our implementation and dataset are here: https://github.com/khangtran2020/pws.
Lay Summary
AI coding assistants like GitHub Copilot are now widely adopted, with over 90% of developers at major U.S. firms using them daily. This widespread adoption is susceptible to a serious security risk: a poisoning attack in which the AI model itself has been secretly tampered with to insert insecure code. However, prior attacks assumed that an outsider could sneak trigger words into a developer's prompts or that the developer would explicitly use them, both of which are unrealistic. PwS instead exploited their coding style as a new and practical attack vector, something developers naturally use to enhance code maintainability. By poisoning a model with high-quality, specifically crafted poisoned datasets, PwS steers it to behave normally on most inputs while silently injecting security flaws whenever it detects a specific formatting style in the code. Evaluation results show that PwS-poisoned models successfully generate vulnerable code 95% of the time while barely affecting normal performance. Crucially, it defeated existing defenses, including safety prompts and state-of-the-art protective fine-tuning. Since code formatting is automatically applied by editor plugins that millions of developers already use, the trigger is essentially invisible. This research urgently highlights security risks and the need for stronger safeguards in the distribution and use of AI coding models.