Toward Cybersecurity-Expert Small Language Models

Large language models (LLMs) are transforming everyday applications, yet they lag behind in specialized fields, such as cybersecurity, due to a lack of high-quality, domain-specific models and training datasets. To address this gap, we present CyberPal 2.0, a family of cybersecurity-expert small language models (SLMs) ranging from 4B–20B parameters. To train CyberPal 2.0, we generate an enriched chain-of-thought cybersecurity instruction dataset built with our data enrichment and formatting pipeline, SecKnowledge 2.0, which integrates expert-in-the-loop steering of reasoning formats alongside LLM-driven multi-step grounding, yielding higher-fidelity, task-grounded reasoning traces for security tasks. Across diverse cybersecurity benchmarks, CyberPal 2.0 consistently outperforms its baselines and matches or surpasses various open and closed-source frontier models, while remaining a fraction of their size. On core threat-investigation tasks, such as correlating vulnerabilities and bug tickets with weaknesses, our best 20B-parameter model outperforms GPT-4o, o1, o3-mini, and Sec-Gemini v1, ranking first, while our smallest 4B-parameter model ranks second. On core cyber threat intelligence knowledge tasks, our models outperform almost all tested frontier models, ranking second only to Sec-Gemini v1. To foster reproducibility and practical adoption, we will release our models as open source.

Cybersecurity teams must analyze large amounts of technical information, such as vulnerability reports, threat intelligence, and security alerts, but today's most powerful AI models are often expensive, difficult to run privately, and not specialized for these workflows. This is a serious limitation because security data is sensitive, and many organizations cannot send it to external AI services. We introduce CyberPal 2.0, a family of smaller language models trained specifically to help analysts understand, explain, and reason about defensive cybersecurity tasks. To build these models, we created SecKnowledge 2.0, a data pipeline that uses cybersecurity experts to define how answers should be structured and then enriches those answers with relevant evidence from documents and search results. This produces training examples that teach the models not only cybersecurity facts, but also how to reason through security questions step by step. Across a broad set of cybersecurity benchmarks, CyberPal 2.0 consistently improves over strong open-source baselines, with average gains of 7–14%. On key cyber threat intelligence tasks, our best model matches or surpasses several much larger frontier models, while remaining small enough to be deployed more practically. These results suggest that carefully designed, expert-guided training data can make compact language models useful specialists for analyst-facing cybersecurity assistance. For cybersecurity teams, this could support more private, affordable, and reliable AI assistants for tasks such as explaining alerts, comparing evidence, summarizing threat reports, and supporting incident-response decisions. Because stronger cybersecurity models can also create misuse risks, we focus on defensive uses and discuss responsible release measures.