DocOS: Towards Proactive Document-Guided Actions in GUI Agents

While Graphical User Interface (GUI) agents have shown promising performance in automated device interaction, they primarily depend on static parametric knowledge from pre-training or instruction tuning. This reliance fundamentally limits their ability to handle long-tailed tasks that require explicit procedural knowledge absent from model parameters, often forcing agents to resort to inefficient and brittle trial-and-error exploration. To mitigate this limitation, we introduce Proactive Document-Guided Action for GUI agents in dynamic, open-web environments, a novel paradigm that mirrors human problem-solving by enabling agents to autonomously search for relevant documentation to resolve long-tailed tasks. To evaluate agents' capability in this paradigm, we propose DocOS, a benchmark designed to assess document-guided problem solving in fully interactive environments. DocOS requires agents to autonomously navigate a web browser, locate relevant online documentation, comprehend procedural instructions, and faithfully ground them into executable GUI actions. Extensive experiments reveal that progress is strictly constrained by dual bottlenecks: agents struggle to reliably locate relevant information during proactive search and frequently fail to faithfully ground retrieved instructions into precise actions, pointing toward document-guided interaction as a crucial pathway for enabling self-evolving GUI agents in dynamic environments.

Modern AI agents can interact with graphical user interfaces (GUIs) such as websites and desktop applications, but they still struggle with unfamiliar or complex tasks. Most existing GUI agents rely mainly on knowledge learned during training, which makes them ineffective when they encounter long-tail tasks that require step-by-step instructions not stored in the model. In these situations, agents often resort to inefficient trial-and-error behavior. To mitigate this limitation, we propose a new framework called Proactive Document-Guided Action. Instead of guessing actions, it actively searches the web for relevant documentation or tutorials, read the instructions, and use them to complete tasks in interactive environments. We also introduce DocOS, a benchmark that evaluates whether agents can successfully find useful documents, understand procedural instructions, and translate them into precise GUI actions.