Dynamic Knowledge Injection for AIXI Agents
Open Access
- 24 March 2024
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (15) , 16388-16397
Abstract
Prior approximations of AIXI, a Bayesian optimality notion for general reinforcement learning, can only approximate AIXI's Bayesian environment model using an a-priori defined set of models. This is a fundamental source of epistemic uncertainty for the agent in settings where the existence of systematic bias in the predefined model class cannot be resolved by simply collecting more data from the environment. We address this issue in the context of Human-AI teaming by considering a setup where additional knowledge for the agent in the form of new candidate models arrives from a human operator in an online fashion. We introduce a new agent called DynamicHedgeAIXI that maintains an exact Bayesian mixture over dynamically changing sets of models via a time-adaptive prior constructed from a variant of the Hedge algorithm. The DynamicHedgeAIXI agent is the richest direct approximation of AIXI known to date and comes with good performance guarantees. Experimental results on epidemic control on contact networks validates the agent's practical utility.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Dynamic Knowledge Injection for AIXI AgentsProceedings of the AAAI Conference on Artificial Intelligence, 2024
- COVID-19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainabilityMathematical Modelling of Natural Phenomena, 2020
- Analysis and Control of Epidemics: A Survey of Spreading Processes on Complex NetworksIEEE Control Systems, 2016
- Epidemic processes in complex networksReviews of Modern Physics, 2015
- The Network Data Repository with Interactive Graph Analytics and VisualizationProceedings of the AAAI Conference on Artificial Intelligence, 2015
- The seven virtues of simple type theoryJournal of Applied Logic, 2008
- Logic for Learning: Learning Comprehensible Theories from Structured DataAssembly Automation, 2004
- Self-similar community structure in a network of human interactionsPhysical Review E, 2003
- A Game of Prediction with Expert AdviceJournal of Computer and System Sciences, 1998
- The context-tree weighting method: basic propertiesIEEE Transactions on Information Theory, 1995