Compositional Self-Improvement

Self-improvement is increasingly important as many tasks become too complex for reliable human annotation. However, it typically fails when a model must solve harder out-of-domain instances in order to generate its own supervision. Many tasks, however, exhibit an underlying compositional structure. We show that when such structure is present, this limitation can be overcome. Our key idea is to construct supervision for complex instances by composing predictions on smaller, in-distribution subproblems, a process we call \emph{compositional self-improvement}. Iterating this process progressively expands the range of instances the model can solve reliably. Empirical results show that the method bypasses direct out-of-distribution generalization by reducing initially out-of-distribution instances to compositions of in-distribution subproblems. We further show that filtering out compositions prone to structured errors does not merely avoid hard cases: the resulting models can still generalize to the filtered-out slices. Compositional self-improvement provides a concrete path to scale beyond the original supervision regime.