DiFR: Inference Verification Despite Nondeterminism

As demand for LLM inference grows, it is becoming increasingly important that providers and their customers can verify that inference processes are performed correctly, without errors or tampering. However, re-running the same inference process twice often leads to different results due to benign numerical noise, making it difficult to distinguish legitimate variation from actual problems. To address this problem, we introduce Token-DiFR (Token-Divergence-From-Reference) a method for verifying open-weight model inference by comparing generated tokens against predictions made by a trusted reference implementation conditioned on the same random seed. Sampling seed synchronization tightly constrains valid outputs, leaving providers minimal room to deviate from correct inference, which allows output tokens themselves to serve as auditable evidence of correctness at zero additional cost to the provider. At temperature zero, Token-DiFR requires no provider cooperation and can be deployed today for auditing inference providers serving any open-weight model. Seed synchronization extends this to non-zero temperatures, enabling a novel capability: retroactive verification of any logged inference request. Token-DiFR reliably identifies sampling errors, simulated bugs, and model quantization, detecting 4-bit quantization with AUC > 0.999 within 300 output tokens. We additionally introduce Activation-DiFR for sample-efficient verification, which uses random projections of internal activations to detect misconfigurations with as few as 2 output tokens while reducing communication overhead by 25--75\% relative to existing methods. We release an open-source integration with vLLM to accelerate practical deployment of verifiable inference.