BioSkillSafety: A Systematic Benchmark for Evaluating Agent Skill Safety in Bioinformatics
Abstract
LLM agents have rapidly emerged as transformative tools for biomedical research, yet their safety risks in bioinformatics-specific contexts remain unexplored. We present BioSkillSafety, the first systematic framework for evaluating skill-based agent safety in bioinformatics domains. Our six-layer taxonomy achieves 100% coverage across 13 attack cases spanning genomics, transcriptomics, clinical, infrastructure, and external communication domains. Through 429 trials across 11 models and 3 real-world skill repositories, we reveal that all skill libraries exhibit consistent vulnerabilities, model safety varies significantly with backbone selection, and domain-specific patterns demand targeted safeguards. These findings establish standardized benchmarks for trustworthy deployment of biomedical AI agents, contributing to safer and more reliable AI-assisted biomedical research.