SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond

The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchmarks often suffer from limited risk coverage and a reliance on subjective evaluation. To address these problems, we introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts. SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement. SafeSciBench distinguishes between safety knowledge and risk to cover extensive scopes and employs objective metrics such as deterministically answerable questions to mitigate evaluation bias. We evaluate 24 advanced LLMs and reveal critical vulnerabilities. We also observe that LLMs exhibit varying degrees of excessive refusal behaviors on safety-related issues. For safety enhancement, we demonstrate that fine-tuning on SafeSciTrain significantly enhances the safety alignment of LLMs. Finally, we argue that determining the safety of a scientific question should depend on the specific context, rather than universally categorizing it as safe or unsafe.

AI language models can now help scientists design molecules, analyze genes, and reason through difficult problems. But the same abilities could help bad actors create dangerous things, such as toxic chemicals, harmful pathogens, or weapons. We need reliable ways to test whether these systems behave safely in scientific settings, yet existing tests cover only narrow topics, often rely on another AI to grade the answers (which introduces bias), and sometimes reuse questions the models already memorized during training. We built SafeSci, a collection of more than 250,000 test questions spanning seven scientific fields. Our central idea is to separate two things that usually get lumped together: knowledge a model should share, like first aid for a chemical spill, versus genuinely harmful requests it should refuse, like instructions for making a toxin. To remove grader bias, we use only questions with clear, checkable answers. We evaluated 24 leading models and created 1.5 million additional examples for training safer ones. We found that today's models are unevenly safe—careful in some areas, lax in others—and sometimes refuse harmless questions out of excess caution. Training on our data measurably improved their safety, pointing toward AI that advances science without making misuse easier.