Should LLM Safety Be More Than Refusing Harmful Instructions?
Preprint
Abstract
This paper presents a systematic evaluation of Large Language Models' (LLMs) behavior on long-tail distributed (encrypted) texts and their safety implications. We introduce a two-dimensional framework for assessing LLM safety: (1) instruction refusal-the ability to reject harmful obfuscated instructions, and (2) generation safety-the suppression of generating harmful responses. Through comprehensive experiments, we demonstrate that models that possess capabilities to decrypt ciphers may be susceptible to mismatched-generalization attacks: their safety mechanisms fail on at least one safety dimension, leading to unsafe responses or over-refusal. Based on these findings, we evaluate a number of pre-LLM and post-LLM safeguards and discuss their strengths and limitations. This work contributes to understanding the safety of LLM in long-tail text scenarios and provides directions for developing robust safety mechanisms.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- A survey on large language model (LLM) security and privacy: The Good, The Bad, and The UglyHigh-Confidence Computing, 2024
- History of Cryptography and Cryptanalysis2024
- SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding2024
- ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs2024
- Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey2024
- Defending ChatGPT against jailbreak attack via self-remindersNature Machine Intelligence, 2023
- TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks2023
- A Deep Bidirectional LSTM-GRU Network Model for Automated Ciphertext ClassificationIEEE Access, 2022
- BPE-Dropout: Simple and Effective Subword Regularization2020
- A Normalized Levenshtein Distance MetricIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007