Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation

Vision–Language Models (VLMs) have shown rapid progress in mobile GUI navigation. This paper presents a systematic study of data scaling, benchmarking, and reasoning for VLM-based agents in this domain. To facilitate rigorous evaluation, we introduce HyperTrack, a large-scale dataset with over 16000 real-world tasks across more than 650 Chinese mobile applications, along with GUIEvalKit, an open-source toolkit for unified benchmarking of VLMs on offline GUI navigation tasks. Using HyperTrack, we analyze the effects of training data scale on both supervised and reinforcement-based finetuning. Our results show that reinforcement-based finetuning consistently outperforms supervised finetuning, particularly in out-of-domain settings, highlighting the synergy between data scaling and reinforcement learning. Leveraging GUIEvalKit, we further benchmark state-of-the-art (SOTA) VLMs and analyze how interaction history and reasoning capabilities influence task completion. Together, HyperTrack and GUIEvalKit provide a comprehensive platform for developing and evaluating VLM agents in mobile GUI navigation tasks.

AI assistants are expected to use mobile apps on behalf of people, but it is difficult to know whether they can reliably read screens, follow instructions, and act across different apps and devices. Existing benchmarks are often small, English-focused, or hard to compare, so researchers lack clear evidence about what makes these agents improve. We built HyperTrack, a collection of more than 16,000 real tasks from Chinese mobile apps, and GUIEvalKit, a toolkit for testing many app-using AI agents under the same rules. With these resources, we studied how performance changes as the amount of training data grows and compared ordinary imitation training with reinforcement-style training, where an agent is rewarded for choosing the right action. We found that performance generally improves with more data, and reinforcement-style training helps most when agents face unfamiliar apps or devices. We also found that adding the model's own recent actions makes offline tests closer to real use, while explicit reasoning can make decisions more varied but less stable. These findings help researchers build and evaluate more reliable mobile AI agents before deployment. They also highlight the need to account for privacy, safety, and misuse risks when releasing datasets and tools for phone automation.