SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMs
Preprint
Abstract
Fine-tuning large language models (LLMs) to adapt to evolving safety policies is costly and impractical. Mechanistic interpretability enables inference-time control through latent activation steering, yet its potential for precise, customizable safety adjustments remains largely untapped. This paper investigates an approach called SafeSteer for guiding the outputs of LLMs by: (i) leveraging category-specific steering vectors for more precise control, (ii) employing a simple, gradient-free unsupervised method to enhance safety steering while preserving text quality, topic relevance, and without explicit refusal, and (iii) accomplishing this without a hard requirement of contrastive pairwise safe data. We also highlight that our method, being simple and effective, aligns with recent studies suggesting that simple techniques often outperform more complex ones in activation steering. We showcase the effectiveness of our approach across various LLMs, datasets, and risk categories, demonstrating its ability to provide precise control, prevent blanket refusals, and guide models toward generating safe content while maintaining topic relevance.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- RewardBench: Evaluating Reward Models for Language Modeling2025
- Spectral Editing of Activations for Large Language Model Alignment2024
- Refusal in Language Models Is Mediated by a Single Direction2024
- HelpSteer 2: Open-source dataset for training top-performing reward models2024
- Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 22024
- Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic2024
- Steering Llama 2 via Contrastive Activation Addition2024
- Parameter-Efficient Detoxification with Contrastive Decoding2024
- Inference-Time Intervention: Eliciting Truthful Answers from a Language Model2023