SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectives

We introduce STEERINGSAFETY, a benchmark for evaluating representation steering methods across nine safety perspectives spanning 18 datasets. While prior work highlights general capabilities of representation steering, we focus on safety perspectives including bias, harmfulness, hallucination, social behaviors, reasoning, epistemic integrity, and normative judgment. Our benchmark provides modularized building blocks for state-of-the-art steering methods, enabling unified implementation of DIM, ACE, CAA, PCA, and LAT with recent enhancements like conditional steering. Results on Gemma-2-2B, Llama-3.1-8B, and Qwen-2.5-7B reveal that strong steering performance depends critically on pairing of method, model, and specific perspective. For instance, DIM shows consistent effectiveness, but all methods exhibit substantial entanglement - where improving effectiveness on one perspective changes performance in other safety perspectives. Social behaviors show highest vulnerability (reaching degradation as high as 76%), jailbreaking often compromises normative judgment such as commonsense morality (degradation up to 26%), and hallucination steering unpredictably shifts political views, from 21% shifts right to 19% shifts to the political left. Our findings underscore the critical need for understanding steering methods from various safety angles.