Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models

Fine-tuned LLMs can covertly encode prompt secrets into outputs via steganographic channels. Prior work demonstrated this threat but relied on trivially recoverable encodings. We formalize payload recoverability via classifier accuracy and show previous schemes achieve 100\% recoverability. In response, we introduce low-recoverability steganography, replacing arbitrary mappings with embedding-space-derived ones. For Llama-8B (LoRA) and Ministral-8B (LoRA) trained on TrojanStego prompts, exact secret recovery rises from 17$\rightarrow$30\% (+78\%) and 24$\rightarrow$43\% (+80\%) respectively, while on Llama-70B (LoRA) trained on Wiki prompts, it climbs from 9$\rightarrow$19\% (+123\%), all while reducing payload recoverability. We then discuss detection. We argue that detecting fine-tuning-based steganographic attacks requires approaches beyond traditional steganalysis. Standard approaches measure distributional shift, which is an expected side-effect of fine-tuning. Instead, we propose a mechanistic interpretability approach: linear probes trained on later-layer activations detect the secret with up to 33\% higher accuracy in fine-tuned models compared to base models, even for low-recoverability schemes. This suggests that malicious fine-tuning leaves actionable internal signatures amenable to interpretability-based defenses.

Suppose a bank, hospital, or other sensitive institution wants to use large language models, but does not trust third-party APIs with private data. A natural solution is to download an open-source LLM, run it on the institution’s own servers, and isolate it from the internet. At first glance, this seems safe, the model can process sensitive data without sending it outside the organization. However, recent work has shown that the model can silently leak data. An attacker can fine-tune an LLM so that it hides private information inside ordinary-looking text that only the attacker knows how to decode. Previous work showed that this kind of attack is possible, but the proposed method was easy to detect. It relied on very simple patterns to leak data into the model’s outputs, which, in practice, makes the attack unrealistic. A defender could quickly identify how and where the secret information was being hidden. In our paper, we improve on this prior work by designing a steganographic scheme based on the model’s internal geometry. In other words, instead of hiding information using an obvious external pattern, we use the structure already present inside the model’s learned representations. This makes it much harder for the defender to tell that a leak is happening. After showing that these attacks can be strengthened, we turn to the question of detection. We find that even when secrets are hidden in seemingly innocent text, the model’s internal representations can still contain traces of the hidden encoding. This means that a defender with full access to the model may be able to detect whether it has learned to leak data. Interestingly, we find that leaking data is especially visible in the later layers of the model. Overall, our work shows that locally deployed LLMs can create new cybersecurity risks, but also that these risks may be detectable by looking inside the model itself.