PinnerFormer
Authors:
Open Access
- 14 August 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3702-3712
Abstract
Sequential models have become increasingly popular in powering personalized recommendation systems over the past several years. These approaches traditionally model a user's actions on a website as a sequence to predict the user's next action. While theoretically simplistic, these models are quite challenging to deploy in production, commonly requiring streaming infrastructure to reflect the latest user activity and potentially managing mutable data for encoding a user's hidden state. Here we introduce PinnerFormer, a user representation trained to predict a user's future long-term engagement using a sequential model of a user's recent actions. Unlike prior approaches, we adapt our modeling to a batch infrastructure via our new dense all-action loss, modeling long-term future actions instead of next action prediction. We show that by doing so, we significantly close the gap between batch user embeddings that are generated once a day and realtime user embeddings generated whenever a user takes an action. We describe our design decisions via extensive offline experimentation and ablations and validate the efficacy of our approach in A/B experiments showing substantial improvements in Pinterest's user retention and engagement when comparing PinnerFormer against our previous user representation. PinnerFormer is deployed in production as of Fall 2021.Keywords
This publication has 20 references indexed in Scilit:
- Self-Attentive Sequential Recommendation2018
- Recurrent Neural Networks with Top-k Gains for Session-based Recommendations2018
- Real-time Personalization using Embeddings for Search Ranking at Airbnb2018
- Graph Convolutional Neural Networks for Web-Scale Recommender Systems2018
- Sequential User-based Recurrent Neural Network Recommendations2017
- Recurrent Recommender Networks2017
- Nonlinear latent factorization by embedding multiple user interests2013
- Factorizing personalized Markov chains for next-basket recommendation2010
- Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language ModelIEEE Transactions on Neural Networks, 2008
- An improved data stream summary: the count-min sketch and its applicationsJournal of Algorithms, 2005