DRIVE: Distributional and Retrieval-Augmented Bidding with Value Evaluation

Auto-bidding is a core component of real-time advertising systems, where decisions must optimize long-term performance under budget and cost constraints, while online exploration is prohibitively risky. Offline reinforcement learning and, more recently, Transformer-based sequence modeling have shown promise for learning bidding policies from logged data, but their unimodal and purely parametric formulations often collapse multiple effective bidding strategies into suboptimal averaged actions and perform unreliably under sparse or long-tail traffic. To mitigate these limitations, we propose DRIVE (Distributional and Retrieval-Augmented Bidding with Value Evaluation), a unified Transformer-based framework that decouples candidate action generation from decision making for offline auto-bidding. DRIVE combines distributional action modeling, retrieval-augmented candidate generation from high-quality historical decisions, and value-based evaluation to select the most promising bid at inference time. Extensive experiments on AuctionNet and additional offline reinforcement learning benchmarks demonstrate that DRIVE consistently improves bidding performance and generalizes well across multiple Transformer–based methods.

Online advertising platforms need to decide how much to bid for each ad impression to get the best results while staying within budgets. Making these decisions in real time is risky because trying out new strategies can be costly. Current methods can learn from past bidding data, but they often oversimplify the problem, averaging multiple good strategies into one and performing poorly when traffic is rare or unusual. We propose DRIVE, which learns to suggest bidding actions more reliably. DRIVE separates generating candidate bids from deciding which one to choose. Our method suggests several possible bids, compares them with high-performing past bids, and picks the best one to maximize performance. This approach helps to pick better bids without risky online experiments. Tests on a large advertising dataset show that DRIVE consistently improves performance and works well across different models. By making bidding more reliable, DRIVE can help online platforms use advertising budgets more effectively and reach their audiences more efficiently.