GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Serving

Large Language Model (LLM)-based agents demonstrate strong reasoning and execution capabilities on complex tasks when guided by structured instructions, commonly referred to as workflows. However, existing workflow-assisted agent serving systems typically rely on predefined templates and shallow matching mechanisms, which limit their ability to capture deep semantic relationships and generalize to previously unseen tasks. To address these limitations, we propose a new workflow management paradigm that represents workflows using a unified graph, termed wGraph, where each node corresponds to an atomic operation. wGraph serves as a shared substrate from which task-specific workflows are dynamically instantiated. Building on wGraph primitives, we introduce GraphFlow, a system that efficiently integrates workflows into agent serving through two key designs. First, adaptive workflow generation dynamically constructs workflows from wGraph based on task semantics and constraint requirements. Second, workflow state management exploits wGraph structure to efficiently manage Key-Value (KV) caches, reducing redundant computation during agent serving. Extensive experiments across five benchmark datasets show that GraphFlow consistently outperforms state-of-the-art methods, yielding an average performance improvement of approximately 4.95 percentage points, while achieving an approximately 4× reduction in memory footprint.

Artificial intelligence (AI) assistants powered by large language models excel at solving complex problems when guided by step-by-step instructions, known as workflows. However, current systems rely on rigid, pre-written instruction templates. This limits their flexibility, causing them to struggle when encountering new or unfamiliar tasks. To overcome this, we developed GraphFlow, a system that transforms workflows from strict lists into a flexible, interconnected map of basic actions. When given a new task, our system acts like a smart navigation app, dynamically charting the best sequence of actions across this map to find a customized solution. Additionally, by understanding the structure of this map, the system efficiently organizes its internal memory. It remembers past steps perfectly, ensuring the AI avoids repeating the same computational work. Our approach makes AI agents significantly smarter and more adaptable. Experiments show it not only improves problem-solving accuracy but also reduces the computer memory required to run these agents by about four times, paving the way for faster, more efficient, and more accessible artificial intelligence.