UNIQ: Conformal Calibration for Adaptive Conservatism in Offline Reinforcement Learning

Offline reinforcement learning requires careful conservatism to counter distribution shift, yet most methods apply a single fixed penalty regardless of how well a given state is covered by the data. We present UnIQ (Uncertainty-Informed Quantile), an offline RL method that adapts its conservatism per-state via conformally calibrated uncertainty. Building on IQL's implicit Q-learning backbone, UnIQ trains a multi-expectile value ensemble, computes distribution-free uncertainty bounds using split conformal prediction, and maps this signal to a state-adaptive expectile tau(s), relaxing conservatism in well-covered regions and strengthening it at the data frontier. On D4RL MuJoCo benchmarks, UnIQ outperforms IQL on Walker2d tasks and replay-heavy settings while operating at near-IQL memory cost (approx. 250 MB peak VRAM)—a 10x reduction versus EDAC. We explicitly report underperforming cases and position UnIQ as a practical mechanism contribution on the performance–efficiency frontier, rather than a claim of overall state-of-the-art.