Haokai Lu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Haokai Lu has a total of 44 co-authors. Here are the top co-authors by number of shared publications:
- Ed H. Chi
Google DeepMind, Mountain View, CA, USA
7 shared publications - Minmin Chen
Google DeepMind, Mountain View, CA, USA
7 shared publications - Jianling Wang
Google DeepMind, Mountain View, CA, USA
5 shared publications - Lexi Baugher
Google, Mountain View, USA
3 shared publications - Shuchao Bi
Google, USA
3 shared publications - Benjamin Lipshitz
Google, Mountain View, USA
2 shared publications
Latest Publications
- Open Access
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 1232-1241
LLM-Based User Personas for Recommendations at Scale
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 1232-1241
Open AccessLLM-Based User Personas for Recommendations at Scale
- Open Access
- 22 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 919-922
Balancing Fine-tuning and RAG: A Hybrid Strategy for Dynamic LLM Recommendation Updates
- 22 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 919-922
Open AccessBalancing Fine-tuning and RAG: A Hybrid Strategy for Dynamic LLM Recommendation Updates
- Open Access
- 8 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 872-877
LLMs for User Interest Exploration in Large-scale Recommendation Systems
Traditional recommendation systems are subject to a strong feedback loop by learning from and reinforcing past user-item interactions, which in turn limits the discovery of novel user interests. To address this, we introduce a hybrid hierarchical framework combining Large Language Models (LLMs) and classic recommendation models for user interest exploration. The framework controls the interfacing between the LLMs and the classic recommendation models through “interest clusters”, the granularity of which can be explicitly determined by algorithm designers. It recommends the next novel interests by first representing “interest clusters” using language, and employs a fine-tuned LLM to generate novel interest descriptions that are strictly within these predefined clusters. At the low level, it grounds these generated interests to an item-level policy by restricting classic recommendation models, in this case a transformer-based sequence recommender to return items that fall within the novel clusters generated at the high level. We showcase the efficacy of this approach on an industrial-scale commercial platform serving billions of users. Live experiments show a significant increase in both exploration of novel interests and overall user enjoyment of the platform.
- 8 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 872-877
Open AccessLLMs for User Interest Exploration in Large-scale Recommendation Systems
Traditional recommendation systems are subject to a strong feedback loop by learning from and reinforcing past user-item interactions, which in turn limits the discovery of novel user interests. To address this, we introduce a hybrid hierarchical framework combining Large Language Models (LLMs) and classic recommendation models for user interest exploration. The framework controls the interfacing between the LLMs and the classic recommendation models through “interest clusters”, the granularity of which can be explicitly determined by algorithm designers. It recommends the next novel interests by first representing “interest clusters” using language, and employs a fine-tuned LLM to generate novel interest descriptions that are strictly within these predefined clusters. At the low level, it grounds these generated interests to an item-level policy by restricting classic recommendation models, in this case a transformer-based sequence recommender to return items that fall within the novel clusters generated at the high level. We showcase the efficacy of this approach on an industrial-scale commercial platform serving billions of users. Live experiments show a significant increase in both exploration of novel interests and overall user enjoyment of the platform.
- Open Access
- 24 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 5723-5730
Multi-Task Neural Linear Bandit for Exploration in Recommender Systems
Exposure bias and its induced feedback loop effect are well-known problems in recommender systems. Exploration is believed to be the key to break such feedback loops. While classical contextual bandit algorithms such as Upper-Confidence-Bound and Thompson Sampling have been successful in addressing the exploration-exploitation trade-off in the single-task settings with one clear reward signal, modern recommender systems often leverage multiple rich sources of feedback such as clicks, likes, dislikes, shares, satisfaction survey responses, and employ multi-task learning in practice. It is unclear how one can incorporate exploration in the multi-task setup with different objectives. In this paper, we study an efficient bandit algorithm tailored to multi-task recommender systems, named Multi-task Neural Linear Bandit (mtNLB). In particular, we investigate efficient feature embeddings in the multi-task setups that could be used as contextual features in the Neural Linear Bandit, a contextual bandit algorithm that nicely combines the representation power from DNN and simplicity in uncertainty calculation from linear models. We further study cost-effective approximations of the uncertainty estimate and principled ways to incorporate uncertainty into the multi-task scoring of items. To showcase the efficacy of our proposed method, we conduct live experiments on a large-scale commercial recommendation platform that serves billions of users. We evaluate the quality of the uncertainty estimate and demonstrate its ability to improve exploration across the different dimensions of the reward signals in comparison to baseline approaches.
- 24 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 5723-5730
Open AccessMulti-Task Neural Linear Bandit for Exploration in Recommender Systems
Exposure bias and its induced feedback loop effect are well-known problems in recommender systems. Exploration is believed to be the key to break such feedback loops. While classical contextual bandit algorithms such as Upper-Confidence-Bound and Thompson Sampling have been successful in addressing the exploration-exploitation trade-off in the single-task settings with one clear reward signal, modern recommender systems often leverage multiple rich sources of feedback such as clicks, likes, dislikes, shares, satisfaction survey responses, and employ multi-task learning in practice. It is unclear how one can incorporate exploration in the multi-task setup with different objectives. In this paper, we study an efficient bandit algorithm tailored to multi-task recommender systems, named Multi-task Neural Linear Bandit (mtNLB). In particular, we investigate efficient feature embeddings in the multi-task setups that could be used as contextual features in the Neural Linear Bandit, a contextual bandit algorithm that nicely combines the representation power from DNN and simplicity in uncertainty calculation from linear models. We further study cost-effective approximations of the uncertainty estimate and principled ways to incorporate uncertainty into the multi-task scoring of items. To showcase the efficacy of our proposed method, we conduct live experiments on a large-scale commercial recommendation platform that serves billions of users. We evaluate the quality of the uncertainty estimate and demonstrate its ability to improve exploration across the different dimensions of the reward signals in comparison to baseline approaches.
- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 726-729
Large Language Models as Data Augmenters for Cold-Start Item Recommendation
The reasoning and generalization capabilities of LLMs can help us better understand user preferences and item characteristics, offering exciting prospects to enhance recommendation systems. Though effective while user-item interactions are abundant, conventional recommendation systems struggle to recommend cold-start items without historical interactions. To address this, we propose utilizing LLMs as data augmenters to bridge the knowledge gap on cold-start items during training. We employ LLMs to infer user preferences for cold-start items based on textual description of user historical behaviors and new item descriptions. The augmented training signals are then incorporated into learning the downstream recommendation models through an auxiliary pairwise loss. Through experiments on public Amazon datasets, we demonstrate that LLMs can effectively augment the training signals for cold-start items, leading to significant improvements in cold-start item recommendation for various recommendation models.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 726-729
Open AccessLarge Language Models as Data Augmenters for Cold-Start Item Recommendation
The reasoning and generalization capabilities of LLMs can help us better understand user preferences and item characteristics, offering exciting prospects to enhance recommendation systems. Though effective while user-item interactions are abundant, conventional recommendation systems struggle to recommend cold-start items without historical interactions. To address this, we propose utilizing LLMs as data augmenters to bridge the knowledge gap on cold-start items during training. We employ LLMs to infer user preferences for cold-start items based on textual description of user historical behaviors and new item descriptions. The augmented training signals are then incorporated into learning the downstream recommendation models through an auxiliary pairwise loss. Through experiments on public Amazon datasets, we demonstrate that LLMs can effectively augment the training signals for cold-start items, leading to significant improvements in cold-start item recommendation for various recommendation models.
- Open Access
- 4 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 1186-1187
Fresh Content Recommendation at Scale: A Multi-funnel Solution and the Potential of LLMs
- 4 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 1186-1187
Open AccessFresh Content Recommendation at Scale: A Multi-funnel Solution and the Potential of LLMs
- Open Access
- 4 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 636-644
Long-Term Value of Exploration: Measurements, Findings and Algorithms
- 4 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 636-644
Open AccessLong-Term Value of Exploration: Measurements, Findings and Algorithms
- Open Access
- 4 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 5082-5091
Fresh Content Needs More Attention: Multi-funnel Fresh Content Recommendation
Recommendation system serves as a conduit connecting users to an incredibly large, diverse and ever growing collection of contents. In practice, missing information on fresh (and tail) contents needs to be filled in order for them to be exposed and discovered by their audience. We here share our success stories in building a dedicated fresh content recommendation stack on a large commercial platform. To nominate fresh contents, we built a multi-funnel nomination system that combines (i) a two-tower model with strong generalization power for coverage, and (ii) a sequence model with near real-time update on user feedback for relevance. The multi-funnel setup effectively balances between coverage and relevance. An in-depth study uncovers the relationship between user activity level and their proximity toward fresh contents, which further motivates a contextual multi-funnel setup. Nominated fresh candidates are then scored and ranked by systems considering prediction uncertainty to further bootstrap content with less exposure. We evaluate the benefits of the dedicated fresh content recommendation stack, and the multi-funnel nomination system in particular, through user corpus co-diverted live experiments. We conduct multiple rounds of live experiments on a commercial platform serving billion of users demonstrating efficacy of our proposed methods.
- 4 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 5082-5091
Open AccessFresh Content Needs More Attention: Multi-funnel Fresh Content Recommendation
Recommendation system serves as a conduit connecting users to an incredibly large, diverse and ever growing collection of contents. In practice, missing information on fresh (and tail) contents needs to be filled in order for them to be exposed and discovered by their audience. We here share our success stories in building a dedicated fresh content recommendation stack on a large commercial platform. To nominate fresh contents, we built a multi-funnel nomination system that combines (i) a two-tower model with strong generalization power for coverage, and (ii) a sequence model with near real-time update on user feedback for relevance. The multi-funnel setup effectively balances between coverage and relevance. An in-depth study uncovers the relationship between user activity level and their proximity toward fresh contents, which further motivates a contextual multi-funnel setup. Nominated fresh candidates are then scored and ranked by systems considering prediction uncertainty to further bootstrap content with less exposure. We evaluate the benefits of the dedicated fresh content recommendation stack, and the multi-funnel nomination system in particular, through user corpus co-diverted live experiments. We conduct multiple rounds of live experiments on a commercial platform serving billion of users demonstrating efficacy of our proposed methods.