BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
Preprint
Abstract
In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety alignment in large language models (LLMs). This dataset uniquely separates annotations of helpfulness and harmlessness for question-answering pairs, thus offering distinct perspectives on these crucial attributes. In total, we have compiled safety meta-labels for 30,207 question-answer (QA) pairs and gathered 30,144 pairs of expert comparison data for both the helpfulness and harmlessness metrics. We further showcase applications of BeaverTails in content moderation and reinforcement learning with human feedback (RLHF), emphasizing its potential for practical safety measures in LLMs. We believe this dataset provides vital resources for the community, contributing towards the safe development and deployment of LLMs. Our project page is available at the following URL: https://sites.google.com/view/pku-beavertails.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- ChatGPT for Robotics: Design Principles and Model AbilitiesIEEE Access, 2024
- TidyBot: personalized robot assistance with large language modelsAutonomous Robots, 2023
- ChatGPT for good? On opportunities and challenges of large language models for educationLearning and Individual Differences, 2023
- The promise of large language models in health careThe Lancet, 2023
- Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language modelsPLOS Digital Health, 2023
- Toxicity in chatgpt: Analyzing persona-assigned language models2023
- Self-Instruct: Aligning Language Models with Self-Generated Instructions2023
- Red Teaming Language Models with Language Models2022
- BBQ: A hand-built bias benchmark for question answering2022
- Reducing Sentiment Bias in Language Models via Counterfactual Evaluation2020