Faithful Mobile GUI Agents with Guided Advantage Estimator

Vision-language model (VLM) based graphical user interface (GUI) agents have shown strong interaction capabilities. However, they often behave unfaithfully, relying on memorized shortcuts rather than grounding actions in displayed screen evidence or user instructions. To address this, we propose Faithful-Agent, a faithfulness-first framework that reformulates GUI interaction to prioritize evidence groundedness and internal consistency. Faithful-Agent employs a two-stage pipeline: (i) a faithfulness-oriented SFT stage to instill abstainment behaviors under evidence perturbations; (ii) an RFT stage that further amplifies faithfulness by introducing the guided advantage estimator (GuAE), an anchor-based and variance-adaptive advantage tempering mechanism built upon GRPO. GuAE prevents advantage collapse in low-variance rollout groups under sparse GUI rewards, and with a thought-action consistency reward, Faithful-Agent (Stage II) elevates the Trap SR from 13.88\% to 80.21\% relative to the baseline, while preserving robust general instruction-following performance.

AI assistants are beginning to use phone apps on behalf of people: opening apps, pressing buttons, and entering text. But a phone screen can be ambiguous. A button may be hidden, the app may not match the user's request, or the assistant may continue with an old plan even after the situation has changed. We study how to make these assistants more faithful to what is actually shown on the screen. Our approach, Faithful-Agent, teaches the assistant to check whether its next action is supported by the current screen and the current instruction. If the evidence is missing or conflicting, it should go back, return home, or stop instead of guessing. This is similar to a careful human user who pauses when the screen does not match what they expected. We find that this training makes AI assistants much more reliable in misleading or incomplete screen situations, while still allowing them to perform ordinary phone tasks well. Our work suggests that useful AI assistants should not only learn how to act, but also when not to act.