MTSTRec: Multimodal Time-Aligned Shared Token Recommender

Sequential recommendation in e-commerce utilizes users' anonymous browsing histories to personalize product suggestions without relying on private information. Existing item ID-based methods and multimodal models often overlook the temporal alignment of modalities like textual descriptions, visual content, and prices in user browsing sequences. To address this limitation, this paper proposes the Multimodal Time-aligned Shared Token Recommender (MTSTRec), a transformer-based framework with a single time-aligned shared token per product for efficient cross-modality fusion. MTSTRec preserves the distinct contributions of each modality while aligning them temporally to better capture user preferences. Extensive experiments demonstrate that MTSTRec achieves state-of-the-art performance across multiple sequential recommendation benchmarks, significantly improving upon existing multimodal fusion. Our code is available at https://github.com/idssplab/MTSTRec.

Online shopping platforms often recommend products to users based on their browsing history, but traditional methods usually rely only on product IDs and ignore important information from other modalities like images, text descriptions, and prices. Even when multimodal data is included, the timing of these elements—like when a user views an image or reads a description—is often misaligned, leading to less accurate recommendations.To solve this, we developed MTSTRec, a new recommendation model that aligns these different types of information, like images, text, and prices, over time. This allows the system to better understand user behavior and make more accurate suggestions.Our experiments show that MTSTRec consistently outperforms other recommendation models, leading to more relevant product suggestions for users. This improvement could make online shopping experiences faster and more personalized, benefiting both customers and retailers. In the future, we plan to adapt MTSTRec for broader applications, unlocking its potential to enhance user experiences and decision-making in multiple domains.