Generative Adversarial Reward Learning for Generalized Behavior Tendency Inference
- 3 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 35 (10) , 9878-9889
Abstract
Recent advances in reinforcement learning have inspired increasing interest in learning user modeling adaptively through dynamic interactions, e.g., in reinforcement learning based recommender systems. In most reinforcement learning applications, reward functions provide the critical guideline for optimization. However, current reinforcement learning-based methods rely on manually-defined reward functions, which cannot adapt to dynamic, noisy environments. Moreover, they generally use task-specific reward functions that sacrifice generalization ability. We propose a generative inverse reinforcement learning for user behavioral preference modeling to address the above issues. Instead of using predefined reward functions, our model can automatically learn the rewards from user's actions based on discriminative actor-critic network and Wasserstein GAN. Our model provides a general approach to characterizing and explaining underlying behavioral tendencies. Our experiments show our method outperforms state-of-the-art methods in several scenarios, namely traffic signal control, online recommender systems, and scanpath prediction.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- Stabilizing Reinforcement Learning in Dynamic Environment with Application to Online Recommendation2018
- DRN2018
- Deep Reinforcement Learning: A Brief SurveyIEEE Signal Processing Magazine, 2017
- Multi-Rate Deep Learning for Temporal Recommendation2016
- Human–Robot InteractionHuman Factors: The Journal of the Human Factors and Ergonomics Society, 2016
- DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving2015
- Collaborative Filtering beyond the User-Item MatrixACM Computing Surveys, 2014
- Context-aware system for proactive personalized service based on context historyExpert Systems with Applications, 2009
- On surrogate loss functions and f-divergencesThe Annals of Statistics, 2009
- Opportunities for multiagent systems and multiagent reinforcement learning in traffic controlAutonomous Agents and Multi-Agent Systems, 2008