Recursive Epistemic Engines for Verifiable Open-Ended Scientific Agents

Current language-model research agents can generate ideas, write code, run experiments, and draft reports, but their unit of progress is still too often a fluent research narrative rather than a verified scientific object. We formalize this gap as the Novelty Horizon: the boundary beyond which an autonomous system cannot jointly generate, execute, verify, and reuse non-derivative hypotheses. We propose Recursive Epistemic Engines (REEs), a template-free architecture that couples large-language-model proposal priors to typed artifact languages, quality-diversity archives, theorem provers, runtime/statistical checkers, and guarded self-modification. The central object is a proof-carrying, experiment-carrying claim package whose value is measured by verified compression gain, expected information gain, transfer, and lineage-auditable reuse. We instantiate the framework with three mechanisms: a verification-gated quality-diversity prover loop; a categorical abstraction layer for compositional scientific objects; and a guarded self-evolution loop for improving verifiers, descriptors, and campaign policies. Three reproducible synthetic experiments isolate the architecture's core claims. Verification-gated QD expands diverse admissible archives while avoiding off-spec artifacts; proof checking without semantic-alignment gates admits many vacuous or misformalized statements; and guarded self-modification preserves heldout utility and verifier strength under Goodhart pressure better than visible-score adoption. The result is a concrete research program for scientific agents that evolve not by producing more plausible papers, but by recursively expanding the space of machine-checkable hypotheses, experiments, and abstractions.