Skip to yearly menu bar Skip to main content


Workshop Sat, Jul 11, 2026 • 8:00 AM – 5:00 PM KST ASEM Ballroom 203

Decision-Making from Offline Datasets to Online Adaptation: Black-Box Optimization to Reinforcement Learning

Aryan Deshwal ⋅ Haruka Kiyohara ⋅ Nghia Hoang ⋅ Tang Thanh Nguyen ⋅ Willie Neiswanger ⋅ Syrine Belakaria ⋅ Jana Doppa
[ OpenReview] 

Abstract

Video

Chat is not available.

Schedule

Time zone: Conference local time zone
8:00 AM
Opening remarks
Jana Doppa
8:15 AM
Invited talk 1: Sergey Levine
Sergey Levine
8:45 AM
Invited talk 2: Aarti Singh
Aarti Singh
9:15 AM
XQCfD: Accelerating Fast Actor-Critic Algorithms with Prior Data and Prior Policies
Daniel Palenicek ⋅ Florian Vogt ⋅ Joe Watson ⋅ Ingmar Posner ⋅ Danica Kragic ⋅ Jan Peters
9:15 AM
Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilities
Georg Martius ⋅ Marco Bagatella ⋅ Thomas Rupf ⋅ Andreas Krause
9:15 AM
Spotlights 1
10:30 AM
Invited talk 3: Clara Wong-Fannjiang
Jacob Gardner
11:00 AM
Rethinking Bayesian Optimization for Co-Optimizing LLM Training Configurations
Zhiliang Chen ⋅ Alfred Leong ⋅ Shao Yong Ong ⋅ Apivich Hemachandra ⋅ Gregory Kang Ruey Lau ⋅ Chuan-Sheng Foo ⋅ Zhengyuan Liu ⋅ Nancy Chen ⋅ Bryan Kian Hsiang Low
11:00 AM
Pitfalls and Remedies for Multi-Task Bayesian Optimization
Carl Hvarfner ⋅ Samuel Daulton ⋅ Maximilian Balandat ⋅ Eytan Bakshy
11:00 AM
Can K Heads Explore Better Than One in Online Reinforcement Learning?
Abhishek Jha ⋅ Satyapragnya Kar ⋅ Kishlay Kumar ⋅ Stephanie Milani ⋅ Rajesh Ranganath
11:00 AM
Cost-Aware Learning
Clara Mohri ⋅ Amir Globerson ⋅ Haim Kaplan ⋅ Tomer Koren ⋅ Yishay Mansour
11:00 AM
Practical Bayesian Optimization for Scientific Discovery
Hamza Chaudhry ⋅ Sean Murphy ⋅ Umesh Padia ⋅ Cengiz Pehlevan ⋅ James Harrison ⋅ George Church ⋅ Jasper Snoek
11:00 AM
Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation
Yu-Du Feng ⋅ Niels Mündler ⋅ Mark Vero ⋅ Martin Vechev
11:00 AM
Orals 1
12:00 PM
Lunch
1:30 PM
The Three Regimes of Offline-to-Online Reinforcement Learning
Lu Li ⋅ Tianwei Ni ⋅ Yihao Sun ⋅ Pierre-Luc Bacon
1:30 PM
Molten Pot: Evaluations & Datasets for Social Offline Reinforcement Learning
Juan Formanek ⋅ Marcel Hedman ⋅ Kale-ab Tessera ⋅ Christopher Holmes
1:30 PM
Orals 2
2:00 PM
Q-Flow: Stable and Expressive Reinforcement Learning with Flow-Based Policy
JaeHyeok Doo ⋅ Byeongguk Jeon ⋅ Seonghyeon Ye ⋅ Kimin Lee ⋅ Minjoon Seo
2:00 PM
Uncertainty-Guided Reward Labeling for Reinforcement Learning under Limited Feedback
Renhao Zhang ⋅ Shreyas Chaudhari ⋅ Bruno C. da Silva
2:00 PM
Abstraction for Offline Goal-Conditioned Reinforcement Learning
Clarisse Wibault ⋅ Alexander D. Goldie ⋅ Antonio León Villares ⋅ Michael A Osborne ⋅ Jakob Foerster
2:00 PM
Meta-GC-TTT: Training Offline Goal-Conditioned Policies for Test-Time Adaptation
Antonio Mari ⋅ Marco Bagatella ⋅ Jonas Hübotter ⋅ Andreas Krause
2:00 PM
Less Tuning, Better Planning: Simplifying Offline Model-Based Planning
Co Yong ⋅ Chan-Hung Yu ⋅ Shao-Hua Sun
2:00 PM
Efficient Off-Policy RL for Video Generation via Forward-Consistent Reward Matching
Hongzheng Yang ⋅ Mengyang LIU ⋅ Haoxuan Wu ⋅ Kun Li ⋅ Yuzhi Zhao ⋅ Wei Liu
2:00 PM
Can We Really Learn One Representation to Optimize All Rewards?
Chongyi Zheng ⋅ Royina Karegoudra Jayanth ⋅ Benjamin Eysenbach
2:00 PM
Boosting for Reinforcement Learning in Structured MDPs
Anh Do ⋅ Jessica Sorrell
2:00 PM
Spotlights 2
4:00 PM
Invited talk 4: Wen Sun Lee
Wen Sun
4:30 PM
Closing Remarks
Receding-Horizon Control via Drifting Models
Alessio Russo ⋅ Daniele Foffano ⋅ Alexandre Proutiere
From Static Policies to Adaptive Priors in Offline Reinforcement Learning
Tianwei Ni ⋅ Vineet Jain ⋅ Akash Karthikeyan ⋅ Pierre-Luc Bacon
Long-Horizon Model-Based Offline Reinforcement Learning Without Explicit Conservatism
Tianwei Ni ⋅ Esther Derman ⋅ Vineet Jain ⋅ Vincent Taboga ⋅ Siamak Ravanbakhsh ⋅ Pierre-Luc Bacon
CoRDE: Concept-Prior Routed Diffusion Experts for Structural Generalization in Robot Manipulation
Haidong Huang ⋅ Xixin Zhao ⋅ Yaohua Zhou ⋅ Jiayi Zhang ⋅ Jun Ma ⋅ Xiaocong Li
When Does Deep RL Beat Calibrated Baselines? A Benchmark Study on Adaptive Resource Control
Guilin Zhang ⋅ Kai Zhao ⋅ Summer Sun ⋅ SHAHRYAR SARKANI ⋅ John M Fossaceca
Toward Deployable Personalization in Robotics: Learning Representative Rewards from Offline Preferences
Taehyung Kim ⋅ Gwangmo Lee ⋅ Jonghak Bae ⋅ dongjae kim ⋅ Jaewoong Han ⋅ Jongeun Choi
Is Temporal-Difference Learning the Only Path to Stitching in RL?
Władysław Pałucki ⋅ Mateusz Ostaszewski ⋅ Benjamin Eysenbach
PubSwap: Public-Data Off-Policy Coordination for Federated RLVR
Anupam Nayak ⋅ Baris Askin ⋅ Muhammed Ustaomeroglu ⋅ Carlee Joe-Wong ⋅ Gauri Joshi
Online Self-Training for Co-Adaptation in Hierarchical Diffusion Policies
Salim Aissi ⋅ Mathilde Kappel ⋅ Olivier Sigaud ⋅ Mohamed Chetouani
Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization
Taeyoung Yun ⋅ Woocheol Shin ⋅ Inhyuck Song ⋅ Jaewoo Lee ⋅ Jinkyoo Park
Learning through Adaptive Queries: a Directional Derivative Approach
Doyoung Heo ⋅ Hwankyu Sung ⋅ Sanghwa Kim ⋅ Seungki Min
Offline Preference Learning with Clustering and Active Data-Augmentation
Jingyuan Liu ⋅ Fatemeh Ghaffari ⋅ Xuchuang Wang ⋅ Xutong Liu ⋅ Mohammad Hajiesmaili ⋅ Carlee Joe-Wong
Mamba as Decision Maker: Exploring Multi-scale Sequence Modeling in Offline Reinforcement Learning
Jiahang Cao ⋅ Ziqing Wang ⋅ Jingkai SUN ⋅ Hao Cheng ⋅ Yecheng Shao ⋅ Wen Zhao ⋅ Gang Han ⋅ Yijie Guo ⋅ Renjing Xu
Can We Really Learn One Representation to Optimize All Rewards?
Chongyi Zheng ⋅ Royina Karegoudra Jayanth ⋅ Benjamin Eysenbach
Adaptive Querying with AI Persona Priors
Kaizheng Wang ⋅ Yuhang Wu ⋅ Assaf Zeevi
FASTER: Value-Guided Sampling for Fast RL
Perry Dong ⋅ Alexander Swerdlow ⋅ Dorsa Sadigh ⋅ Chelsea Finn
A Planning-Based Reinforcement Learning Approach to Numerical Optimization
Uma Maheswari Natarajan ⋅ Sai Shruti Prakhya ⋅ Shivani S Shukla ⋅ Chandramouli Kamanchi ⋅ Raghuram Bharadwaj Diddigi
OrthoBO: Orthogonalized Bayesian Hyperparameter Optimization
Maresa Schröder ⋅ Pascal Janetzky ⋅ Michael Klar ⋅ Stefan Feuerriegel
On the Role of Proposal Support in Diffusion-Based Offline RL for Sequential Decision-Making
Yunho Kim ⋅ JAEHYUN PARK ⋅ Sejin Kim ⋅ Byung-Jun Lee ⋅ Sundong Kim
REVES: REvision and VErification–Augmented Training for Test-Time Scaling
Yuanxin Liu ⋅ Ruida Zhou ⋅ Xinyan Zhao ⋅ Amr Sharaf ⋅ Hongzhou Lin ⋅ Arijit Biswas ⋅ Zhaoran Wang ⋅ Mingyi Hong
CODA: Coordination via On-Policy Diffusion for Multi-Agent Offline Reinforcement Learning
Marcel Hedman ⋅ Juan Formanek ⋅ Anya Sims ⋅ Riccardo Zamboni ⋅ Trevor McInroe ⋅ John Torr ⋅ Elliot Fosong
Molten Pot: Evaluations & Datasets for Social Offline Reinforcement Learning
Juan Formanek ⋅ Marcel Hedman ⋅ Christopher Holmes
Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilities
Marco Bagatella ⋅ Thomas Rupf ⋅ Georg Martius ⋅ Andreas Krause
Unified Latent Steering and Residual Refinement for Online Improvement of Diffusion Policy Models
Zhengbang Zhu ⋅ Ziyan Li ⋅ Xiu Yuan ⋅ Hanbo Zhang ⋅ Yuxiao Liu ⋅ Chongjie Zhang ⋅ Yong Yu ⋅ Weinan Zhang ⋅ Minghuan Liu
Combinatorial Allocation Bandits with Nonlinear Arm Utility
Yuki Shibukawa ⋅ Koichi Tanaka ⋅ Yuta Saito ⋅ Shinji Ito
A Mutual Information Lower Bound for Multimodal Regression Active Learning
Leonardo Ferreira Guilhoto ⋅ Akshat Kaushal ⋅ Paris Perdikaris
On the Optimal Sample Complexity of Offline Multi-Armed Bandits with KL Regularization
Kaixuan Ji ⋅ Qiwei Di ⋅ Heyang Zhao ⋅ Qingyue Zhao ⋅ Quanquan Gu
Learning Reasoning Rewards from Expert Demonstrations with Inverse Reinforcement Learning
Claudio Fanconi ⋅ Nicolás Astorga ⋅ Mihaela van der Schaar
MIRT: Multi-Dimensional IRT for SLO-Adaptive Multi-Agent Routing
Hak Hyun Kim ⋅ Benjamin Huh ⋅ Jimin Moon ⋅ Chia-Wei Lee ⋅ Jason Peng ⋅ Wesley J. Marrero ⋅ Soroush Vosoughi
Abstraction for Offline Goal-Conditioned Reinforcement Learning
Clarisse Wibault ⋅ Antonio León Villares ⋅ Jakob Foerster
The Three Regimes of Offline-to-Online Reinforcement Learning
Lu Li ⋅ Tianwei Ni ⋅ Yihao Sun ⋅ Pierre-Luc Bacon
Adaptive Stratified Active Statistical Inference
Pinaki Mohanty ⋅ Rajiv Khanna
Reward-Wise Value Estimation for Multi-Reward Optimization in Large Language Models
Quan Wei ⋅ Zhongruo Wang ⋅ Chenliang Li ⋅ Xi Chen ⋅ Oana Frunza ⋅ Mingyi Hong
V-VLAPS: Value-Guided Planning for Vision-Language-Action Models
Ke Ren ⋅ Ali Salamatian ⋅ Kieran Pattison ⋅ Cyrus Neary
BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks
Zhiliang Chen ⋅ Apivich Hemachandra ⋅ Bryan Kian Hsiang Low
Freeze the Policy, Infer the Goal: Cross-Domain Imitation with World Models
Xingyuan Zhang ⋅ Marvin Alles ⋅ Patrick van der Smagt ⋅ Philip Becker-Ehmck
Online Regret Minimization in Linear Bandits with Offline data.
Sushant Vijayan ⋅ Karthikeyan Shanmugam ⋅ Arun Sai Suggala ⋅ Soumyabrata Pal
Bayesian Optimization with Early Trial Termination for Speeding Up Parallel Neural Network Training
Apivich Hemachandra ⋅ Yizhan Han ⋅ See-Kiong Ng ⋅ Bryan Kian Hsiang Low
Rethinking Bayesian Optimization for Co-Optimizing LLM Training Configurations
Zhiliang Chen ⋅ Alfred Leong ⋅ Shao Yong Ong ⋅ Apivich Hemachandra ⋅ Gregory Kang Ruey Lau ⋅ Chuan-Sheng Foo ⋅ Zhengyuan Liu ⋅ Nancy Chen ⋅ Bryan Kian Hsiang Low
Dual Advantage Fields
Alexey Zemtsov ⋅ Maksim Bobrin ⋅ Alexander Nikulin ⋅ Fakhri Karray ⋅ Vladislav Kurenkov ⋅ Arip Asadulaev
VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA Models
Damir Shodiev ⋅ Aleksei Staroverov ⋅ Nikita Kachaev ⋅ Aleksey Kovalev ⋅ Aleksandr Panov
Fairness of Exposure in Stochastic Multiple-play Multi-armed Bandits
Youngmi Jin ⋅ Dongdeok Kim ⋅ Young-Joo Suh
How Many Initial Points Does Bayesian Optimization Need?
Mujin Cheon ⋅ Dong-Yeun Koh ⋅ Calvin Tsay
AsyncOPD: How Stale Can On-Policy Distillation Be?
Wonjun Kang ⋅ Kevin Galim ⋅ Seunghyuk Oh ⋅ Minjun Kang ⋅ Sanghyun Park ⋅ Donghoon Kim ⋅ Minseo Kim ⋅ Rishabh Tiwari ⋅ Yuchen Zeng ⋅ HYUNG IL KOO ⋅ Kangwook Lee
Receding-Horizon Execution for Action Chunking in Offline-to-Online Reinforcement Learning
Jihwan Lee ⋅ Geonwoo Cho ⋅ Hojun Yi ⋅ Jaegyun Im ⋅ Giseung Park ⋅ Sundong Kim
A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise
Mattia Forzo ⋅ Alessio Russo ⋅ Enea Monzio Compagnoni ⋅ Aldo Pacchiano
In-context Learning for Latent Space Bayesian Optimization
Harri Lähdesmäki ⋅ Julien Martinelli
Pitfalls and Remedies for Multi-Task Bayesian Optimization
Carl Hvarfner ⋅ Samuel Daulton ⋅ Maximilian Balandat ⋅ Eytan Bakshy
Policy-Only Power Sampling for Vision-Language-Action Control
Jimin Park ⋅ Wonjeong Choi ⋅ Jaekyun Moon
In-Context Pure Exploration in Continuous Decision Spaces
Alessio Russo ⋅ Yin-Ching Lee ⋅ Ryan Welch ⋅ Aldo Pacchiano
Learning to Orchestrate Heterogeneous Agents under Uncertainty
Mary Chriselda Antony Oliver ⋅ Lan Jiang ⋅ Aaron B Anampiu ⋅ Elaf Almahmoud ⋅ Francesco Quinzan ⋅ Umang Bhatt
Aligning Flow Map Policies with Optimal $Q$-Guidance
Christos Ziakas ⋅ Alessandra Russo ⋅ Joey Bose
Future Information-Directed Sampling for Bayesian Nonstationary Bandits
Yichen Song ⋅ Alessio Russo ⋅ Aldo Pacchiano
Meta-GC-TTT: Training Offline Goal-Conditioned Policies for Test-Time Adaptation
Antonio Mari ⋅ Marco Bagatella ⋅ Jonas Hübotter ⋅ Andreas Krause
AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
Shaolong Chen ⋅ Madalina Ciobanu ⋅ Qingqing Mao ⋅ Ritankar Das
Structured Behavioral Heterogeneity as Latent Regime Constraints
Yuting Yan ⋅ Haozhou Gao ⋅ Xinye Chen ⋅ Yinghao Fu ⋅ Shuang Li
Forgetting to Improve: Principled Data Removal in Active Learning
Manuel Wendl ⋅ Erik Englesson ⋅ Andreas Krause ⋅ Carl Henrik Ek
Sampling-Based Safe Reinforcement Learning
Luca Vignola ⋅ Bruce D Lee ⋅ Manish Prajapat ⋅ Manuel Wendl ⋅ Andreas Krause ⋅ Yarden As
Learning Insider-Threat Intervention Policies from Offline Logs
Cyrine Fekih ⋅ Ashish Sai ⋅ Adriana Iamnitchi ⋅ Dennis J. N. J. Soemers
Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning
Abhinav Anand ⋅ mingze wu ⋅ Shweta Verma ⋅ Mira Mezini
Efficient Off-Policy RL for Video Generation via Forward-Consistent Reward Matching
Hongzheng Yang ⋅ Mengyang LIU ⋅ Haoxuan Wu ⋅ Yuzhi Zhao ⋅ Wei Liu
Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding
Hoang Phan ⋅ Minh Pham ⋅ Chau Pham ⋅ Chinmay Hegde ⋅ Trung Le ⋅ Qi Lei
FICReg: Forward-Inverse Consistency Regularization for Latent World Models
Sungwon Seo ⋅ Jean Seong Bjorn Choe ⋅ Jong-Kook Kim
From Offline Global Information to Online Decentralized Policies in Edge Network Scheduling
Justin Chang ⋅ Angela Zhang ⋅ Jiayi Chen ⋅ Aditya Akella
CombiLatent: Neural Combinatorial Optimization via Latent Space Search under Sinkhorn Divergence Regularization
Younes Boukacem ⋅ Hodhaifa Benouaklil ⋅ Ait S Yassine ⋅ Houssem Talbi ⋅ Mehdi Z Adjal ⋅ Faissal IZERMINE
Towards Adapting Contrastive RL to the Offline Setting
Catherine Ji ⋅ Grace Tan ⋅ Benjamin Eysenbach
Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement Learning
Mahmoud Selim ⋅ Cristina Cipriani ⋅ Karl Johansson
Q-Flow: Stable and Expressive Reinforcement Learning with Flow-Based Policy
JaeHyeok Doo ⋅ Seonghyeon Ye ⋅ Kimin Lee ⋅ Minjoon Seo
Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation
Yu-Du Feng ⋅ Niels Mündler ⋅ Mark Vero ⋅ Martin Vechev
Efficient Algorithms for Contextual Apple Tasting with Log-Loss
Byeongwoo An ⋅ Kapilan Balagopalan ⋅ sehwa jeong ⋅ Jiun Jeong ⋅ Kyoungseok Jang ⋅ Hyowon Wi ⋅ Noseong Park ⋅ Gi-Soo Kim ⋅ Kwang-Sung Jun
Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Control
Ruslan Rakhimov ⋅ George Bredis ⋅ Yuriy Maksyuta ⋅ Daniil Gavrilov
Cost-Aware Learning
Clara Mohri ⋅ Amir Globerson ⋅ Haim Kaplan ⋅ Tomer Koren
XQCfD: Accelerating Fast Actor-Critic Algorithms with Prior Data and Prior Policies
Daniel Palenicek ⋅ Florian Vogt ⋅ Joe Watson ⋅ Ingmar Posner ⋅ Danica Kragic ⋅ Jan Peters
LLM-PriorCB: Textual Contextual Bandits with LLM-Induced Priors
Geon-Hyeong Kim ⋅ Yu Jin Kim ⋅ June Yong Yang ⋅ Woohyung Lim ⋅ Youngsoo Jang ⋅ Moontae Lee
Uncertainty-Guided Reward Labeling for Reinforcement Learning under Limited Feedback
Renhao Zhang ⋅ Shreyas Chaudhari ⋅ Bruno C. da Silva
Improving Multi-Agent Coordination with a Drift-Aware RL Objective
Sangeun Park ⋅ Guhyeon Kang ⋅ Minhae Kwon
Trust the Batch, Online or Offline: Adaptive Policy Optimization for Post-Training
Rasool Fakoor ⋅ Murdock Aubry ⋅ Nicholas Stranges ⋅ Alex Smola
Good Experience Maximization
Anthony GX-Chen ⋅ Stephanie Milani ⋅ Jatin Prakash ⋅ Sumit Chopra ⋅ Rob Fergus ⋅ Rajesh Ranganath
Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors
Guozheng Ma ⋅ Lu Li ⋅ Zilin Wang ⋅ Pierre-Luc Bacon ⋅ Dacheng Tao
What Makes Value Learning Efficient in Residual Reinforcement Learning?
Guozheng Ma ⋅ Lu Li ⋅ Haoyu Wang ⋅ Zixuan Liu ⋅ Pierre-Luc Bacon ⋅ Dacheng Tao
Practical Bayesian Optimization for Scientific Discovery
Hamza Chaudhry ⋅ Sean Murphy ⋅ Umesh Padia ⋅ Cengiz Pehlevan ⋅ James Harrison ⋅ George Church ⋅ Jasper Snoek
Can K Heads Explore Better Than One in Online Reinforcement Learning?
Abhishek Jha ⋅ Satyapragnya Kar ⋅ Kishlay Kumar ⋅ Stephanie Milani ⋅ Rajesh Ranganath