Budget-Constrained Step-Level Diffusion Caching

Step-level caching accelerates diffusion models by exploiting temporal redundancy across denoising steps. Existing methods make per-step cache decisions using threshold-based heuristics, without directly optimizing for final output quality. As a result, their inference latency varies across inputs and is difficult to control at deployment. In this work, we propose BudCache, which inverts this formulation: rather than letting per-step error thresholds dictate the runtime cost, we fix the compute budget in advance and search for the cache policy that best preserves the final output. To tackle the combinatorial complexity of step selection, we combine Simulated Annealing with deterministic Hill Climbing. This offline search identifies high-quality cache policies within minutes and introduces no online search or thresholding overhead during inference. When the compute budget is very tight, we further introduce cache-aware schedule alignment, which adapts the time discretization to the selected cache policy to reduce cache-induced trajectory mismatch. Experiments on FLUX.1-dev and Wan2.1 show that BudCache achieves better generation quality than heuristic caching baselines under the same inference budgets.

Step-level caching makes diffusion models faster by reusing intermediate results across nearby generation steps. Existing caching methods usually decide whether to reuse or recompute based on hand-designed thresholds during generation, which makes the actual runtime vary across inputs and makes deployment harder to control. BudCache instead fixes the computation budget in advance and searches offline for which generation steps should be recomputed and which can safely reuse cached information. This produces a static caching plan with predictable runtime and no extra online decision cost. For very tight computation budgets, BudCache also adjusts the generation schedule to better match the selected caching plan. Experiments on image and video generation models show that BudCache preserves visual quality better than existing heuristic caching methods under the same inference budgets.