RLxF: RL from World Feedback

Reinforcement Learning from Feedback (RLxF) has emerged as a central paradigm for aligning and improving modern learning systems, with Reinforcement Learning from Human Feedback (RLHF) playing a pivotal role in the success of large language models and other foundation models. However, an exclusive focus on human-provided signals fundamentally constrains scalability, objectivity, and task coverage. Many real-world objectives—such as energy efficiency, system health, latency, throughput, economic return, and long-term sustainability—are not best captured by subjective human judgments, but instead manifest directly through measurable interactions with the environment. This workshop, RLxF: Reinforcement Learning from World Feedback, argues that the next phase of RLxF must move beyond human feedback toward richer, grounded, and often automated signals arising from the world itself. The workshop aims to unify emerging research that leverages world feedback: signals generated by physical, digital, economic, or social systems that reflect the consequences of an agent’s actions. These include, but are not limited to, resource consumption, safety metrics, performance traces, biological or health indicators, market responses, and multi-agent dynamics. We seek to explore methodological questions such as how to model noisy, delayed, and partially observed world feedback; how to combine heterogeneous feedback sources with human input; and how generative models, simulators, and diffusion-based policies can enable learning from sparse or implicit signals. By framing world feedback as a first-class training signal, the workshop connects reinforcement learning, control, foundation models, systems research, and embodied AI under a common conceptual lens. By bringing together researchers from machine learning, robotics, systems, economics, and AI alignment, RLxF aims to catalyze a shift from human-centric to world-grounded learning paradigms. We believe this perspective is essential for deploying learning systems that are scalable, robust, and meaningfully aligned with real-world objectives. The workshop will feature invited talks, contributed papers, and open discussions to surface shared challenges, benchmark opportunities, and future research directions. Ultimately, RLxF: RL from World Feedback seeks to redefine how feedback is conceptualized in reinforcement learning, positioning interaction with the world—not just human preference—as a core driver of intelligent behavior.