FLIPS: Instance-Fingerprinting for LLMs via Pseudo-random Sequences

Literature reveals that a Large Language Model's (LLM) behavior is not only conditioned by its original weights but also its instance-level parameters, such as instructional prompt, sampling configuration or quantization. A model that generates safe outputs under one configuration may produce toxic content under another. However, current LLM identification techniques (such as fingerprinting) focus on intellectual property protection, and their design favors robustness to changes in these instance-level parameters. This poses a critical challenge for AI regulation in which compliance assessments target actual deployed behaviors, not model provenance. In this paper, we introduce instance-level fingerprinting, a regulator-oriented paradigm that distinguishes configurations of the same LLM. Our method FLIPS, exploits biases in generated binary random sequences to reach 96% (closed-set) and 90% (open-set, where some targets are unknown) identification accuracy across 237 model instances, versus 35% for the adapted LLMmap baseline. This shows that instance-level fingerprinting is both necessary for regulation and practically feasible. Code available at https://github.com/GurvanR/FLIPS-LLM-Instance-Fingerprinting.

We evaluate AI language models with benchmarks that map their behavior, but a gap opens between test and reality. Commercial models are updated often, sometimes silently, and can be altered in undocumented ways. Also, one model sold under a single name may secretly run in many configurations, such as a hidden instruction that reshapes its behavior. Depending on that setup, the same model can amplify biases like racism or political slant, or have its safety guardrails badly weakened. So how can a regulator confirm, without repeating a costly audit, that a deployed model matches a version they approved? Tools that recognize a model from its behavior, called fingerprinting, were built for copyright protection, to see through such changes and prove one model copies another. By design they ignore configuration: what was noise becomes the very signal a regulator needs. We are the first to reframe fingerprinting for regulation: our method matches a deployed model to a library of previously audited models from a handful of questions, with no special access. This makes routine checks practical and flags the moment a model no longer matches anything on file.