Improving LLM-Based Recommenders with Conservative Generative Flow Networks

Generative Flow Networks (GFlowNets) have recently been used to improve diversity and mitigate popularity bias in LLM-based recommender systems, yet most objectives are developed under online-style assumptions. In offline LLM-based recommendation, learning is constrained to a fixed logged dataset, yielding partial support over token transitions on the dataset-induced token-prefix DAG. Naively applying Sub-Trajectory Balance (SubTB) becomes non-identifiable and can arbitrarily allocate probability mass to unsupported regions. We formalize this failure and identify three sources of non-identifiability that induce distributional shift between the dataset-implied policy and the learned policy: (i) flow overestimation, (ii) forward mass leakage, and (iii) backward compensation. To address it, we propose CFlower, which introduces a conservative SubTB objective that explicitly penalizes unsupported forward flow mass, and combines it with dataset-constrained policy learning with on-policy sampling on the dataset-induced DAG for efficient training under offline constraints. Experiments on three Amazon recommendation datasets show that CFlower improves distributional matching and delivers a stronger accuracy--exposure trade-off than prior GFlowNet and SFT baselines, while serving as a more reliable reference policy for downstream RL fine-tuning.

Large language models are becoming a powerful tool for recommendation systems, helping platforms decide what products, videos, or articles to show people next. However, when these systems are trained only on past user activity, they often become too focused on already popular items and may make unreliable choices beyond what the data truly supports. This can reduce diversity and make it harder for less popular but still relevant items to be recommended. Earlier methods such as Flower were developed to make recommendation systems fairer and less dominated by already popular items. This is important because systems that only chase accuracy often keep showing the same popular choices, limiting what users discover and reducing exposure for other relevant items. Our work addresses a different but important problem: when these methods are trained only on fixed historical data, they can drift away from the patterns that the data truly supports, leading to unreliable recommendations. We propose CFlower, a more cautious training method that keeps the model closer to trustworthy evidence in the data while preserving the goal of broader and fairer recommendations. Experiments on three Amazon datasets show that our approach produces a better balance between recommendation quality and fairer exposure across items.