Rapid Fixes, Gradual Failures: Exploring Iterative Self-Correction Dynamics in Large Language Models for Program Synthesis

Achieving one-shot success in code generation remains challenging for large language models (LLMs). Iterative self-repair, in which models refine their own programs in response to feedback, has been proposed as a potential remedy. However, its effectiveness for programming tasks is not well understood. We study the dynamics of this loop across three frontier LLMs (GPT-4, GPT-5.1, Claude Sonnet 4.5), four execution-based benchmarks (HumanEval, MBPP+, APPS-Intro, APPS-Comp), two languages (Python and Java), and two history conditions (h = 0: critique only; h = 1: full refinement history). Three findings emerge. First, recovery is overwhelmingly a task-level property. Failing seeds on tasks where at least one other initial sample already passed are 36× more likely to recover than failing seeds on tasks where every initial sample failed. Second, refinement is heavily front-loaded: 53% of the total pass-fraction gain occurs at attempt 1. Recovered and non-recovered seeds diverge immediately (a 46 pp gap in pass fraction by attempt 1). A substantial fraction of non-recovered seeds (~38% at h = 0, ~49% at h = 1) reproduce the same wrong program for ten attempts. Finally, adding the full refinement history does not improve recovery rate (p > 0.35) and on harder benchmarks actively degrades it (-7.5 pp on APPS-Comp).