Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization

Proactive large language model (LLM) agents aim to actively plan, query, and interact over multiple turns, enabling efficient task completion beyond passive instruction following and making them essential for real-world, user-centric applications. Agentic reinforcement learning (RL) has recently emerged as a promising solution for training such agents in multi-turn settings, allowing interaction strategies to be learned from feedback. However, existing pipelines face a critical challenge in balancing task performance with user engagement, as passive agents can not efficiently adapt to users' intentions while overuse of human feedback reduces their satisfaction. To address this trade-off, we propose BAO, an agentic RL framework that combines behavior enhancement to enrich proactive reasoning and information-gathering capabilities with behavior regularization to suppress inefficient or redundant interactions and align agent behavior with user expectations. We evaluate BAO on multiple tasks from the UserRL benchmark suite, and demonstrate that it substantially outperforms RL baselines under controlled comparisons, while achieving comparable or even superior performance to frontier LLM agents, highlighting its effectiveness for training proactive, user-aligned LLM agents in complex multi-turn scenarios.

Large language model (LLM) agents are becoming more interactive, allowing them to ask questions, gather information, and plan over multiple steps. However, agents that ask too few questions may fail tasks, while agents that ask too many can burden users. We introduce Behavior Agentic Optimization (BAO), a training framework that teaches AI agents to interact more efficiently and avoid unnecessary or repetitive user interactions. Experiments on challenging multi-turn tasks show that BAO improves both task success and user experience, helping agents solve problems more effectively with less user effort.