FinRegGuard: Can LLMs Detect Regulatory Evasion? A 100K Adversarial Benchmark for U.S. Financial Law

Large language models are increasingly deployed in financial compliance workflows, yet no benchmark evaluates their robustness against adversarial regulatory evasion attempts. We introduce FinRegGuard, a 100K-sample adversarial benchmark spanning 75 U.S. regulatory documents across eight domains, generated via a novel Vision-Language Adversarial Red-Teaming (VLART) pipeline. Each sample includes an evasion query, forensic reasoning, a compliant response, and evasion steps—enabling both classification and preference-based training. We evaluate five open-source LLMs (1.5B–7B) under seven methods: zero-shot, few-shot, GRR zero-shot, supervised fine-tuning (SFT), GRR-SFT, DPO, and GRR-DPO, using imbalance-aware metrics with bootstrap confidence intervals on dual test sets (balanced and natural distribution). We find that (1) LLMs fail at regulatory evasion detection out-of-the-box (best prompting MCC = 0.443); (2) SFT yields large gains (Gemma3-4B: MCC 0.126→0.581); (3) structured regulatory reasoning underperforms plain SFT at this scale despite producing interpretable citations; and (4) DPO preference alignment severely degrades compliance performance relative to SFT (average MCC 0.112 vs. 0.434), a failure mode with direct practical implications for compliance system design. We release the dataset, code, and model checkpoints to support research in trustworthy AI for financial compliance.