DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion

Diffusion Transformer models can generate images with remarkable fidelity and detail, yet training them at ultra-high resolutions remains extremely costly due to the self-attention mechanism's quadratic scaling with the number of image tokens. In this paper, we introduce Dynamic Position Extrapolation (DyPE), a novel, training-free method that enables pre-trained diffusion transformers to synthesize images at resolutions far beyond their training data, with no additional sampling cost. DyPE takes advantage of the spectral progression inherent to the diffusion process, where low-frequency structures converge early, while high-frequencies take more steps to resolve. Specifically, DyPE dynamically adjusts the model's positional encoding at each diffusion step, matching their frequency spectrum with the current stage of the generative process. This approach allows us to generate images at resolutions that exceed the training resolution dramatically, e.g., 16 million pixels using FLUX. On multiple benchmarks, DyPE consistently improves performance and achieves state-of-the-art fidelity in ultra-high-resolution image generation, with gains becoming even more pronounced at higher resolutions.

Modern AI models can generate stunning, realistic images from text descriptions. However, training these models to create ultra-high-resolution images requires an immense amount of computing power and memory due to their architecture. If we try to force a standard model to generate an image much larger than what it was trained on, it gets "confused" about where things belong, resulting in strange, repeating visual artifacts as well as blurry images. To solve this, we developed a training-free method called Dynamic Position Extrapolation (DyPE). In the image generation process, broad shapes always form first, followed by fine textures later. DyPE acts like a dynamic camera lens that adjusts its positional focus to match this progression. Early on, it helps the model understand the overall layout of the large canvas. As the image becomes clearer, it shifts to focus purely on generating sharp, fine details. This approach allows existing, pre-trained models to generate massive images, exceeding 16 million pixels, without needing any costly retraining or extra generation time. Ultimately, this framework makes state-of-the-art, high-resolution image synthesis significantly more accessible and computationally efficient.