Modeling diverse preferences in movie artwork personalization with large language models

Large language models (LLMs) have demonstrated remarkable success in various recommendation applications. A key challenge within these systems is respecting diverse user preferences and value rather than offering a one-size-fits-all solution. Our work focuses on pluralistic preference alignment of LLMs for artwork recommendation on entertainment platforms, where systems need to consider diverse cultural norms, social values, and individual preferences when engaging with users. On these platforms, users typically interact with an extensive catalog of titles, each represented by specific artwork. Just as users' tastes are multi-faceted, titles contain varied themes and tones that may appeal to different viewers based on their values. Given this heterogeneity, we explore the novel problem of personalizing artwork recommendations using LLMs. For example, the same title might feature both heartfelt family drama and intense action scenes; a user preferring romantic content may favor artwork emphasizing emotional warmth, whereas a user preferring thrillers may find high-intensity scenes more appealing. We post-train 3B and 8B Llama3 models to select the optimal artwork for a given title-user pair based on the specific user's preferences. Our experiments with 110K training data and 5K held-out test data show that post-training yields a 3-5\% improvement over a non-LLM production baseline. Overall, our work suggests a promising direction for hyper-personalized artwork recommendations, extending beyond text-based recommendation tasks and providing a pathway for pluralistic alignment with visual data.