Skip to yearly menu bar Skip to main content


Workshop Thu, Jul 9, 2026 • 4:00 PM – 1:00 AM PDT GRAND BALLROOM 101-102

RLxF: RL from World Feedback

Shao-Hua Sun ⋅ Xingyou Song ⋅ Akari Asai ⋅ Yash Akhauri ⋅ Chenglei Si ⋅ Tengyu Ma ⋅ Theresa Eimer
Project Page [ OpenReview] 

Abstract

Video

Chat is not available.

Schedule

Timezone: America/Los_Angeles
4:00 PM
Opening Remarks
Shao-Hua Sun
6:00 PM
Coffee Break
8:00 PM
Lunch
10:00 PM
Poster Session 2
11:00 PM
Coffee Break
12:00 AM
Panel discussion
Shao-Hua Sun ⋅ Xingyou Song ⋅ Roberta Raileanu ⋅ Jesse Zhang ⋅ Yilun Du ⋅ Brian Zhan
→ Reinforcement Learning from Rich Feedback with Distributional DAgger
Rishabh Agrawal ⋅ Jacob Fein-Ashley ⋅ Paria Rashidinejad
→ Bootstrapping Language and Numerical Feedback for Reinforcement Learning in LLMs
Ang Li ⋅ Yifei Wang ⋅ Zhihang Yuan ⋅ Stefanie Jegelka ⋅ Yisen Wang
→ ECHO: Terminal Agents Learn World Models for Free
Vaishnavi Shrivastava ⋅ Ahmed Awadallah ⋅ Dimitris Papailiopoulos
→ Right in the Right Way: Combining Verifiable Rewards with Human Demonstrations
Mehul Damani ⋅ Isha Puri ⋅ Idan Shenfeld ⋅ Jacob Andreas
→ ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions
Chuanyang Jin ⋅ Binze Li ⋅ Haopeng Xie ⋅ Cathy Mengying Fang ⋅ Tianjian Li ⋅ Shayne Longpre ⋅ Hongxiang Gu ⋅ Maximillian Chen ⋅ Tianmin Shu
→ Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents
Changdae Oh ⋅ Wendi Li ⋅ Seongheon Park ⋅ Min-Hsuan Yeh ⋅ Tanwi Mallick ⋅ Sharon Li
→ Single-Step Initialization for Exploratory Parallel Rollouts in Diffusion LLMs
Dongjae Jeon ⋅ Bumjun Kim ⋅ Mingyu Kim ⋅ Albert No
→ Reinforcing VLAs in Task-Agnostic World Models
Yucen Wang ⋅ Rui Yu ⋅ Fengming Zhang ⋅ Junjie Lu ⋅ Xinyao Qin ⋅ Tianxiang Zhang ⋅ Kaixin Wang ⋅ Li Zhao
→ Robust Exploration through Generative Replay
Inhyuck Song ⋅ Taeyoung Yun ⋅ Jaewoo Lee ⋅ Jinkyoo Park
→ Coherent Off-Policy Improvement of Large Behaviour Models with Learned Rewards
Christian Scherer ⋅ Joe Watson ⋅ Daniel Palenicek ⋅ Theo Gruner ⋅ Ingmar Posner ⋅ Jan Peters
→ What Drives Interactive Improvement from Feedback?
Bartłomiej Cupiał ⋅ Jan Łojek ⋅ Mikołaj Garstecki ⋅ Szymon J Pobłocki ⋅ Alicja Ziarko ⋅ Piotr Milos
→ Learning to Ask May Be Better Than Learning to Answer
Asmi Kumar ⋅ Chuang Gan ⋅ Zhang-Wei Hong
→ MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference
Raphaël Baur ⋅ Yannick Metz ⋅ Maria Gkoulta ⋅ Mennatallah El-Assady ⋅ Giorgia Ramponi ⋅ Thomas Kleine Buening
→ Learning Diffusion Planners from World Feedback: A No-Go Result on Bit-Exact Safety Rewards and an ODD-Adaptive Shared/Expert Decomposition
Yun Li ⋅ Ehsan Javanmardi ⋅ Yidu Zhang ⋅ Simon Thompson ⋅ Qunli Zhang ⋅ Zifan Zeng ⋅ Shiming Liu ⋅ Peng Wang ⋅ Zixuan Guo ⋅ Manabu Tsukada
→ Self-Improving World Modelling with Latent Actions
Yifu QIU ⋅ Zheng Zhao ⋅ Waylon Li ⋅ Yftah Ziser ⋅ Anna Korhonen ⋅ Shay Cohen ⋅ Edoardo Ponti
→ Prompt-Driven Exploration
Sunshine Jiang ⋅ John Marangola ⋅ David Zhang ⋅ Raghuram Kowdeed ⋅ Ruiyang Luo ⋅ Nitish Dashora ⋅ Richard Li ⋅ Pulkit Agrawal ⋅ Zhang-Wei Hong
→ Training Language Agents to Learn from Experience
Yuval Shalev ⋅ Zifeng Ding ⋅ Mateja Jamnik
→ Multi-Objective Reinforcement Learning for Large-Scale Tote Allocation in Human-Robot Collaborative Fulfillment Centers
Sikata Sengupta ⋅ Guangyi Liu ⋅ Omer Gottesman ⋅ Joseph Durham ⋅ Michael Kearns ⋅ Aaron Roth ⋅ Michael Caldara
→ FutureSim: Replaying World Events to Evaluate Adaptive Agents
Shashwat Goel ⋅ Nikhil Chandak ⋅ Arvindh Arun ⋅ Ameya Pandurang Prabhu ⋅ Steffen Staab ⋅ Moritz Hardt ⋅ Maksym Andriushchenko ⋅ Jonas Geiping
→ Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models
Samy Jelassi ⋅ Mujin Kwun ⋅ Rosie Zhao ⋅ Yuanzhi Li ⋅ Nicolò Fusi ⋅ Yilun Du ⋅ Sham Kakade ⋅ Carles Domingo i Enrich
→ RL Excursions during Pre-training: How early is too early for on-policy learning?
Rachit Bansal ⋅ Clara Mohri ⋅ Tian Qin ⋅ David Alvarez-Melis ⋅ Sham Kakade
→ RAGEN-2: Reasoning Collapse in Agentic RL
Zihan (Zenus) Wang ⋅ Chi Gui ⋅ Xing Jin ⋅ Qineng Wang ⋅ Licheng Liu ⋅ Kangrui Wang ⋅ Shiqi Chen ⋅ Linjie Li ⋅ Zhengyuan Yang ⋅ Pingyue Zhang ⋅ Yiping Lu ⋅ Jiajun Wu ⋅ Li Fei-Fei ⋅ Lijuan Wang ⋅ Yejin Choi ⋅ Manling Li
→ Workflow-R1: Group Sub-sequence Policy Optimization for Multi-turn Workflow Construction
Mingze Kong ⋅ Zikun Qu ⋅ Zhongquan Zhou ⋅ Pengyu Liang ⋅ Xiang Li ⋅ Zhiwei Shang ⋅ Zhi Hong ⋅ Kaiyu Huang ⋅ Zhiyong Wang ⋅ Zhongxiang Dai
→ Code2World: A GUI World Model via Renderable Code Generation
Yuhao Zheng ⋅ Li'an Zhong ⋅ Yi Wang ⋅ Rui Dai ⋅ Kaikui Liu ⋅ Xiangxiang Chu ⋅ Linyuan Lü ⋅ Phil Torr ⋅ Kevin Qinghong Lin
→ Towards Budget-Aware Agents: Can LLM Agents Know What They Will Spend?
Yuxiang Lin ⋅ Zihan (Zenus) Wang ⋅ Mengyang Liu ⋅ Yuxuan Shan ⋅ Longju Bai ⋅ Junyao Zhang ⋅ Xing Jin ⋅ Boshan Chen ⋅ Jinyan Su ⋅ Xingyao Wang ⋅ Jiaxin Pei ⋅ Manling Li
→ Critic-Guided Reinforcement Unlearning in Text-to-Image Diffusion
Mykola Vysotskyi ⋅ Zakhar Kohut ⋅ Mariia Shpir ⋅ Taras Rumezhak ⋅ Volodymyr Karpiv
→ Counterfactual Transport Flows for Offline Conservative Trajectory Refinement
Lena Krieger ⋅ Xuan Zhao ⋅ Zhuo Cao ⋅ Qin Wang ⋅ Hanno Scharr ⋅ Ira Assent
→ The Interplay of Harness Design and Post-Training in LLM Agents
Kyungmin Kim ⋅ Youngbin Choi ⋅ Seoyeon Lee ⋅ Suhyeon Jun ⋅ Dongwoo Kim ⋅ Sangdon Park
→ Real-world Reinforcement Learning from Suboptimal Interventions
yinuo zhao ⋅ Huiqian Jin ⋅ Lechun Jiang ⋅ Xinyi Zhang ⋅ Kun Wu ⋅ Pei Ren ⋅ Zhiyuan Xu ⋅ Zhengping Che ⋅ Junjie Ji ⋅ Lei Sun ⋅ Dapeng Wu ⋅ Chi Harold Liu ⋅ Jian Tang
→ Aligning Language Models from User Interactions
Thomas Kleine Buening ⋅ Jonas Hübotter ⋅ Barna Pasztor ⋅ Idan Shenfeld ⋅ Giorgia Ramponi ⋅ Andreas Krause
→ Rank-Then-Act: Reward-Free Control from Frame-Order Progress
Yuriy Maksyuta ⋅ George Bredis ⋅ Ruslan Rakhimov ⋅ Daniil Gavrilov
→ Making Execution Time a Trainable Reward for Code Generation
Pierre Chambon ⋅ Kunhao Zheng ⋅ Juliette Decugis ⋅ Benoît Sagot ⋅ Gabriel Synnaeve
→ Unified Noise Steering for Efficient Human-Guided VLA Adaptation
Junjie Lu ⋅ Xinyao Qin ⋅ Yuhua Jiang ⋅ Tianxiang Zhang ⋅ Xiaoyu Chen ⋅ Kaixin Wang ⋅ Chuheng Zhang ⋅ Bin Liang ⋅ Jun Yang ⋅ Min Xu ⋅ Li Zhao
→ ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents
Yuanzheng Zhu ⋅ Mingyu Zhao ⋅ Chinmay Savadikar ⋅ Han Li ⋅ Shuang Xie ⋅ Alberto Castelo ⋅ Tianfu Wu ⋅ Lingyun Wang
→ NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Models
Wen Huang ⋅ Haoran Sun ⋅ Yongjian Guo ⋅ Yunxuan Ma ⋅ Haoran Li ⋅ Jing Long ⋅ Zhouying Mo ⋅ Guan Zhong ⋅ Yucheng Guo ⋅ shuai di ⋅ Xiong J Wu
→ A Predictive Law for On-Policy Self-Distillation From World Feedback
Tommy He ⋅ Jerome Sieber ⋅ Matteo Saponati
→ On-Policy Self-Distillation via Prompt Optimization
Jongho Park ⋅ Donghyun Lee ⋅ Matei Zaharia ⋅ Jason Lee
→ Learning to Ideate for Scientific Impact
Shubham Kale ⋅ Aniketh Garikaparthi ⋅ Manasi Patwardhan
→ Learning Stateful Predictive Knowledge From Experience
Yan Song ⋅ Xidong Feng ⋅ Bo Liu ⋅ Xinyu Cui ⋅ Zichen Liu ⋅ Haotian Fu ⋅ Mengyue Yang ⋅ Cheng Deng ⋅ Jian Zhao ⋅ Jun Wang
→ Multi-Rollout On-Policy Distillation via Peer Successes and Failures
Weichen Yu ⋅ Xiaomin Li ⋅ Yizhou Zhao ⋅ Xiaoze Liu ⋅ RUOWANG ZHANG ⋅ Haixin Wang ⋅ Yinyi Luo ⋅ Chen Wu ⋅ Gaurav Mittal ⋅ Matt Fredrikson ⋅ Yu Hu
→ Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
Yinghui He ⋅ Simran Kaur ⋅ Adithya Bhaskar ⋅ Yongjin Yang ⋅ Jiarui Liu ⋅ Narutatsu Ri ⋅ Liam Fowl ⋅ Abhishek Panigrahi ⋅ Danqi Chen ⋅ Sanjeev Arora
→ A Few Teacher Steps Go a Long Way: Cost-Efficient On-Policy Data Augmentation for Agent Post-Training
Junze Ye ⋅ Jiayi Cheng ⋅ Lu Miao ⋅ Michal Mankowski ⋅ Jose Blanchet ⋅ Mohsen Bayati
→ Learning Visually-Grounded Active View Selection for Embodied Question Answering
Juil Koo ⋅ Daehyeon Choi ⋅ Sangwoo Youn ⋅ Phillip Y Lee ⋅ Minhyuk Sung
→ Can We Really Learn One Representation to Optimize All Rewards?
Chongyi Zheng ⋅ Royina Karegoudra Jayanth ⋅ Benjamin Eysenbach
→ SignalBench: Comparing Dense Feedback Methods for Long-Horizon Agents
Sergio Hernández-Gutiérrez ⋅ Matteo Merler ⋅ Ilze Amanda Auzina ⋅ Joschka Strüber ⋅ Ameya Pandurang Prabhu ⋅ Matthias Bethge
→ MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
Wenhan Ma ⋅ Jianyu Wei ⋅ Liang Zhao ⋅ Hailin Zhang ⋅ Bangjun Xiao ⋅ Lei Li ⋅ Qibin Yang ⋅ Bofei Gao ⋅ Yudong Wang ⋅ Rang Li ⋅ Jinhao Dong ⋅ Fuli Luo ⋅ Zhifang Sui
→ Bridging Physical Reasoning and Task Generalization via Visual Action Outcome Reasoning Alignment
Han-Jun Ko ⋅ Jr-Jen Chen ⋅ Haobo Yuan ⋅ Hsin-Ying Lee ⋅ Tiancheng SHEN ⋅ Ming-Hsuan Yang ⋅ Yu-Chiang Wang
→ AutoOR: Scalably Post-training LLMs to Autoformalize Operations Research Problems
Sumeet Motwani ⋅ Chuan Du ⋅ Aleksandar Petrov ⋅ Christopher E Davis ⋅ Phil Torr ⋅ Antonio R Papania-Davis ⋅ Weishi Yan
→ EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models
Perry Dong ⋅ Kuo-Han Hung ⋅ Tian Gao ⋅ Dorsa Sadigh ⋅ Chelsea Finn
→ ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Emergent Adaptation
Jingqi Zhou ⋅ Sheng Wang ⋅ Dezhao Deng ⋅ Junwen Lu ⋅ Junwei Su ⋅ Qintong Li ⋅ Jiahui Gao ⋅ Hao Wu ⋅ Jiyue Jiang ⋅ Lingpeng Kong ⋅ Dunhong Jin ⋅ Chuan Wu
→ Transferability for General Reasoning: An Automated Curriculum for Multi-Domain LLM RL
Yongjin Yang ⋅ Jiarui Liu ⋅ Yinghui He ⋅ Lechen Zhang ⋅ Bernhard Schölkopf ⋅ Zhijing Jin
→ World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments
Ananya Mantravadi ⋅ Harshit Rajgarhia ⋅ Prasanna Desikan ⋅ Abhishek Mukherji
→ When Does Non-Uniform Replay Matter in Reinforcement Learning?
Michal Korniak ⋅ Mikołaj Czarnecki ⋅ Yarden As ⋅ Piotr Milos ⋅ Pieter Abbeel ⋅ Michal Nauman
→ Training AI Co-Scientists using Rubric Rewards
Shashwat Goel ⋅ Rishi Hazra ⋅ Dulhan Jayalath ⋅ Timon Willi ⋅ Parag Jain ⋅ Shen ⋅ Ilias Leontiadis ⋅ Francesco Barbieri ⋅ Yoram Bachrach ⋅ Jonas Geiping ⋅ Chenxi Whitehouse
→ Playing with Fire : What Transfers When RL Trains a Language Agent?
Kaousheik Jayakumar ⋅ Mahesh Ramesh ⋅ Hemanth Ram ⋅ Pavan Thodima ⋅ Ramani Duraiswami ⋅ Dinesh Manocha ⋅ Aniket Rege ⋅ Emmanouil-Vasileios Vlatakis-Gkaragkounis
→ Learning, Fast and Slow: Towards LLMs That Adapt Continually
Rishabh Tiwari ⋅ Kusha Sareen ⋅ Lakshya A Agrawal ⋅ Joseph E Gonzalez ⋅ Matei Zaharia ⋅ Kurt Keutzer ⋅ Inderjit Dhillon ⋅ Rishabh Agarwal ⋅ Fnu Devvrit
→ PubSwap: Public-Data Off-Policy Coordination for Federated RLVR
Anupam Nayak ⋅ Baris Askin ⋅ Muhammed Ustaomeroglu ⋅ Carlee Joe-Wong ⋅ Gauri Joshi
→ VLFEEDBACK-EEG: Neural Signals as Implicit Feedback for Vision-Language Model Alignment
Angela Lopez Cardona ⋅ Sebastian Idesis ⋅ Mireia M Bruns ⋅ Matteo Mazzini ⋅ Joemon Jose ⋅ Sergi Abadal ⋅ Ioannis Arapakis
→ Learn Where Outcomes Diverge: Efficient VLA RL via Probabilistic Chunk Masking
Vaidehi Bagaria ⋅ Nikshep Grampurohit ⋅ Pulkit Verma