Human-powered AI Gym: Lessons Learned as the Test and Evaluation Team for the DARPA SHADE Program: Human-powered AI Gym
Open Access
- 17 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-5
Abstract
We present the best practices and collaborative methods used during the DARPA SHADE program, exploring the use of AI agents to advise diplomatic negotiations. In this accelerated 18-month program, we created an environment for seven research performers to collaborate in the development of an AI Gym. This AI Gym provided an area in which AI developed by the various teams could compete, negotiate, and explain their intentions as they played the game of Diplomacy, allowing us to evaluate their abilities against other digital and human agents.Keywords
Funding Information
- Defense Advanced Research Projects Agency
This publication has 4 references indexed in Scilit:
- Scalable Scientific Interactive Research Computing With Project ScincoComputing in Science & Engineering, 2023
- Human-level play in the game ofDiplomacyby combining language models with strategic reasoningScience, 2022
- Tapis: An API Platform for Reproducible, Distributed Computational Research2021
- It Takes Two to Lie: One to Lie, and One to Listen2020