Foveate, Attribute, and Rationalize: Towards Safe and Trustworthy AI
Preprint
Abstract
Users' physical safety is an increasing concern as the market for intelligent systems continues to grow, where unconstrained systems may recommend users dangerous actions that can lead to serious injury. Covertly unsafe text, language that contains actionable physical harm, but requires further reasoning to identify such harm, is an area of particular interest, as such texts may arise from everyday scenarios and are challenging to detect as harmful. Qualifying the knowledge required to reason about the safety of various texts and providing human-interpretable rationales can shed light on the risk of systems to specific user groups, helping both stakeholders manage the risks of their systems and policymakers to provide concrete safeguards for consumer safety. We propose FARM, a novel framework that leverages external knowledge for trustworthy rationale generation in the context of safety. In particular, FARM foveates on missing knowledge in specific scenarios, retrieves this knowledge with attribution to trustworthy sources, and uses this to both classify the safety of the original text and generate human-interpretable rationales, combining critically important qualities for sensitive domains such as user safety. Furthermore, FARM obtains state-of-the-art results on the SafeText dataset, improving safety classification accuracy by 5.29 points.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Risk-graded Safety for Handling Medical Queries in Conversational AI2022
- SafeText: A Benchmark for Exploring Physical Safety in Language Models2022
- SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems2022
- Generated Knowledge Prompting for Commonsense Reasoning2022
- Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPTransactions of the Association for Computational Linguistics, 2021
- Confronting Abusive Language Online: A Survey from the Ethical and Human Rights PerspectiveJournal of Artificial Intelligence Research, 2021
- Zero-shot Fact Verification by Claim Generation2021
- Detecting Cyberbullying and Cyberaggression in Social MediaACM Transactions on the Web, 2019
- A Knowledge-Grounded Neural Conversation ModelProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Approaches to Automated Detection of Cyberbullying: A SurveyIEEE Transactions on Affective Computing, 2017