Changing the Environment Based on Empowerment as Intrinsic Motivation
Open Access
Abstract
One aspect of intelligence is the ability to restructure your own environment so that the world you live in becomes more beneficial to you. In this paper we investigate how the information-theoretic measure of agent empowerment can provide a task-independent, intrinsic motivation to restructure the world. We show how changes in embodiment and in the environment change the resulting behaviour of the agent and the artefacts left in the world. For this purpose, we introduce an approximation of the established empowerment formalism based on sparse sampling, which is simpler and significantly faster to compute for deterministic dynamics. Sparse sampling also introduces a degree of randomness into the decision making process, which turns out to beneficial for some cases. We then utilize the measure to generate agent behaviour for different agent embodiments in a Minecraft-inspired three dimensional block world. The paradigmatic results demonstrate that empowerment can be used as a suitable generic intrinsic motivation to not only generate actions in given static environments, as shown in the past, but also to modify existing environmental conditions. In doing so, the emerging strategies to modify an agent’s environment turn out to be meaningful to the specific agent capabilities, i.e., de facto to its embodiment.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Causal Entropic ForcesPhysical Review Letters, 2013
- Hierarchical curiosity loops and active sensingNeural Networks, 2012
- Empowerment for continuous agent—environment systemsAdaptive Behavior, 2011
- Evolutionary Theory and the Ultimate–Proximate Distinction in the Human Behavioral SciencesPerspectives on Psychological Science, 2011
- Keep Your Options Open: An Information-Based Driving Principle for Sensorimotor SystemsPLOS ONE, 2008
- INFORMATION FLOWS IN CAUSAL NETWORKSAdvances in Complex Systems, 2008
- Intrinsic Motivation Systems for Autonomous Mental DevelopmentIEEE Transactions on Evolutionary Computation, 2007
- What is intrinsic motivation? A typology of computational approachesFrontiers in Neurorobotics, 2007
- Intrinsic and Extrinsic Motivations: Classic Definitions and New DirectionsContemporary Educational Psychology, 2000
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948