PPT-Eval: A Benchmark for Computer-Use Agents on PowerPoint Tasks

Creating and editing slides is a rich, multimodal activity that is ubiquitous in professional and educational settings, making it an ideal testbed for real-world computer-use agents. Microsoft PowerPoint is among the most widely adopted and feature-rich environments for presentation creation. We introduce PPT-Eval, a benchmark of 120 PowerPoint tasks across 12 files that cover both content creation and presentation editing scenarios, organized by difficulty. A central challenge in this domain is evaluation: tasks are complex, multimodal, and often admit many valid solutions. Moreover, today’s agents frequently make only partial progress, which binary success metrics fail to capture. To address this, we design a robust evaluation framework to help create task-specific rubrics for PowerPoint tasks, taking inspiration from and building on past works for rubric-based evaluation. These rubrics award partial credit for intermediate steps, penalize unnecessary changes and poor aesthetics, and provide natural language feedback. This nuanced approach proves highly effective, achieving a Kendall's $\tau_b$ correlation of 0.77 with human judgments. We find that existing frontier agents still struggle with solving PowerPoint tasks, with strong models like Claude-4.5-Opus achieving only a 45\% success rate and an average partial score of 57\%.

Agents that operate computers to perform tasks are becoming ubiquitous and powerful. In particular, with vision language models (VLMs), agents can now interact with the computer's graphical user interface using actions like humans would (click, scroll, type, etc.) Creating and editing slides is an ideal testbed to benchmark such computer-use GUI agents given the rich, multimodal operations that presentation creation involves as well as the importance of presentation creation across professional and educational settings. In this work, we introduce PPT-Eval, a benchmark of 120 PowerPoint tasks across 12 files that cover both content creation and presentation editing scenarios, organized by difficulty. A central challenge in this domain is evaluation: tasks are complex, multimodal, and often admit many valid solutions. Moreover, today’s agents frequently make only partial progress, which binary success metrics fail to capture. To address this, we design a robust evaluation framework to help create task-specific rubrics for PowerPoint tasks, taking inspiration from and building on past works for rubric-based evaluation. These rubrics award partial credit for intermediate steps, penalize unnecessary changes and poor aesthetics, and provide natural language feedback. We find that the best computer-use GUI agents still have a huge gap in performance compared to human users (45\% vs. 80\% success rate), making PPT-Eval a strong benchmark to evaluate progress of computer-use agents.