PSBench: Editing Image via GUI Agents in Photoshop

Photoshop is a professional image editing software whose complex multi-level menus, fine-grained operations, and layer-based non-destructive editing pose substantial challenges for automated agents. Existing GUI benchmarks and methods primarily target web interfaces and short-horizon, low-complexity tasks, falling short in modeling the multi-step decision-making and semantic understanding required by professional graphic software. We introduce PSBench, the first benchmark specifically designed for image editing in Adobe Photoshop, consisting of 600 human-annotated tasks across three difficulty levels, with tasks drawn from official tutorials and popular real-world workflows. PSBench covers core functionalities such as canvas adjustment, layer manipulation, and filter application, and provides fine-grained evaluation metrics tailored to each task category. Our experiments show that even the state-of-the-art system, Agent S3, achieves a success rate of only 18.09\% on difficult tasks, indicating that GUI agents still face considerable challenges in operating complex professional software. Furthermore, human-in-the-loop evaluations reveal that MLLMs, when serving as interactive assistants, can significantly improve novice users’ task completion rates and reduce operation time.

Many people hope that artificial intelligence can one day use software in the same way humans do, automatically completing complex tasks such as editing images, filling out forms, or assisting with digital workflows. However, most existing research focuses on relatively simple environments such as web browsers, while much less attention has been given to professional software. Programs like Photoshop are especially challenging because they contain complex menus, layered editing systems, and many operations that require precise actions and specialized knowledge. To study whether AI systems can effectively use professional image editing software, we introduce PSBench, the first benchmark specifically designed for Photoshop. PSBench contains 600 realistic image editing tasks covering common workflows such as cropping, layer editing, and filter usage. Using this benchmark, we evaluated several state-of-the-art AI systems and found that even the strongest current methods struggle with difficult Photoshop tasks. At the same time, we discovered that AI systems are often good at understanding editing goals and planning meaningful action sequences. When AI acts as an assistant that provides step-by-step guidance while humans perform the actual operations, users complete tasks more successfully and much more efficiently. These findings suggest that for complex professional software, human-AI collaboration may be a more practical and impactful direction than fully autonomous AI agents.