Going Beyond Black-Box Models by Leveraging Behavioral Insights: an Intent-Aware Multi-Stage Recommendation Framework

Most platforms today rely on large-scale machine learning (ML) models to predict user choices. However, these systems are black-box in nature, lacking the understanding of the underlying data generation process (DGP), or why the user likes a certain product. In this work, we demonstrate that incorporating the data generation process into black-box ML can be leveraged to improve these black-box ML systems. In the context of recommender systems, it has been established that the data generation process is intent-driven decision making. However, explicit intent labels are hard to get. At the same time, latent intent works exist start from HMMs, but intent obtained from those are not interpretable. In this work, we propose to mine user behaviors to obtain intent predictions and a principled approach to incorporate the data generation process into large-scale multi-stage recsys. We propose IA-Recsys, a three-stage whole-page recommendation framework that aligns a multi-stage industrial size recsys with the underlying intent-based DGP of a recsys. IA-recsys first predicts explicit user intent, then incorporate into reward model and generate whole-page reccommendations that cover a full set of intents. The IA-Recsys framework was validated through extensive A/B testing on YouTube, the world’s largest video recommendation platform. By incorporating intents related to novelty and familiarity, it achieved a 0.05\% increase in daily active users (DAU), one of the most significant business metric improvements observed in recent YouTube experiments.