JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (26) , 27419-27427
Abstract
Large Language Models (LLMs) aligned with human feedback have recently garnered significant attention. However, it remains vulnerable to jailbreak attacks, where adversaries manipulate prompts to induce harmful outputs. Exploring jailbreak attacks enables us to investigate the vulnerabilities of LLMs and further guides us in enhancing their security. Unfortunately, existing techniques mainly rely on handcrafted templates or generated-based optimization, posing challenges in scalability, efficiency and universality. To address these issues, we present JailPO, a novel black-box jailbreak framework to examine LLM alignment. For scalability and universality, JailPO meticulously trains attack models to automatically generate covert jailbreak prompts. Furthermore, we introduce a preference optimization-based attack method to enhance the jailbreak effectiveness, thereby improving efficiency. To analyze model vulnerabilities, we provide three flexible jailbreak patterns. Extensive experiments demonstrate that JailPO not only automates the attack process while maintaining effectiveness but also exhibits superior performance in efficiency, universality, and robustness against defenses compared to baselines. Additionally, our analysis of the three JailPO patterns reveals that attacks based on complex templates exhibit higher attack strength, whereas covert question transformations elicit riskier responses and are more likely to bypass defense mechanisms.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Jailbreaking Black Box Large Language Models in Twenty Queries2025
- SimPO: Simple Preference Optimization with a Reference-Free Reward2024
- Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking2024
- OpenAGI: When LLM Meets Domain Experts2023
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model2023
- Multi-step Jailbreaking Privacy Attacks on ChatGPT2023
- Red Teaming Language Models with Language Models2022
- Rank Analysis of Incomplete Block Designs: I. The Method of Paired ComparisonsBiometrika, 1952