Confident Action Decision via Hierarchical Policy Learning for Conversational Recommendation
Open Access
- 30 April 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1386-1395
Abstract
Conversational recommender systems (CRS) aim to acquire a user’s dynamic interests for a successful recommendation. By asking about his/her preferences, CRS explore current needs of a user and recommend items of interest. However, previous works may not determine a proper action in a timely manner which leads to the insufficient information gathering and the waste of conversation turns. Since they learn a single decision policy, it is difficult for them to address the general decision problems in CRS. Besides, existing methods do not distinguish whether the past behaviors inferred from the historical interactions are closely related to the user’s current preference. To address these issues, we propose a novel Hierarchical policy learning based Conversational Recommendation framework (HiCR). HiCR formulates the multi-round decision making process as a hierarchical policy learning scheme, which consists of both a high-level policy and a low-level policy. In detail, the high-level policy aims to determine what type of action to take, such as a recommendation or a query, by observing the comprehensive conversation information. According to the decided action type, the low-level policy selects a specific action, such as which attribute to ask or which item to recommend. The hierarchical conversation policy enables CRS to decide an optimal action, resulting in reducing the unnecessary consumption of conversation turns and the continuous failure of recommendations. Furthermore, in order to filter out the unnecessary historical information when enriching the current user preference, we extract and utilize the informative past behaviors that are attentive to the current needs. Empirical experiments on four real-world datasets show the superiority of our approach against the current state-of-the-art methods.Keywords
This publication has 35 references indexed in Scilit:
- Interactive Path Reasoning on Graph for Conversational Recommendation2020
- Towards Question-based Recommender Systems2020
- LightGCN2020
- Interactive Recommender System via Knowledge Graph-enhanced Reinforcement Learning2020
- Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems2020
- Towards Knowledge-Based Recommender Dialog System2019
- Towards Conversational Search and Recommendation2018
- OpenKE: An Open Toolkit for Knowledge Embedding2018
- Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning2017
- Towards Conversational Recommender Systems2016