VENOMREC: Cross-Modal Interactive Poisoning for Targeted Promotion in Multimodal LLM Recommender Systems
Abstract
Multimodal large language models (MLLMs) are pushing recommender systems (RecSys) toward content-grounded retrieval and ranking via cross-modal fusion. We find that while cross-modal consensus often mitigates conventional poisoning that manipulates interaction logs or perturbs a single modality, it also introduces a new attack surface where synchronised multimodal poisoning can reliably steer fused representations along stable semantic directions during fine-tuning. To characterise this threat, we formalise cross-modal interactive poisoning and propose VENOMREC, which performs Exposure Alignment to identify high-exposure regions in the joint embedding space and Cross-modal Interactive Perturbation to craft attention-guided coupled token--patch edits. Experiments on four real-world multimodal datasets demonstrate that VENOMREC consistently outperforms strong baselines, achieving 0.73 mean ER@20 and improving over the strongest baseline by +0.52 absolute ER points on average, while maintaining comparable recommendation utility. Code is available at https://github.com/GuoweiGuan666/VenomRec.
Lay Summary
When you shop online or scroll through short videos, the AI behind your recommendations increasingly "reads" both an item's picture and its description together. It has been widely assumed this is harder to manipulate, because if a seller tampers with the text, the picture should give it away, and vice versa. We show that this assumption fails. We built a method called VenomRec that edits an item's picture and its description at the same time, making tiny changes that quietly agree with each other. It first locates the "popular" region of the recommender's internal map, then nudges the item there in a coordinated way. On real Amazon and short-video datasets, this reliably pushed unpopular items to the top of recommendations while leaving normal users' experience unchanged, and standard defenses failed to catch it. The very feature meant to make these systems trustworthy, namely agreement between picture and text, can be turned into a tool for manipulation.