REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations

Large language models (LLMs) achieve strong performance across many tasks but remain vulnerable to hallucinations, making it important to systematically evaluate their reliability under realistic adversarial inputs. We formulate hallucination elicitation as a constrained optimization problem, where the goal is to find semantically coherent adversarial prompts that are equivalent to benign user prompts. Existing attack methods remain limited: discrete prompt-based attacks preserve semantic equivalence and coherence but search only over a limited set of prompt variations, while continuous latent-space attacks explore a richer space but often decode into prompts that are no longer valid rephrasings. To address these limitations, we propose REALISTA, a realistic latent-space attack framework. REALISTA constructs an input-dependent dictionary of valid editing directions, each corresponding to a semantically equivalent and coherent rephrasing, and optimizes continuous combinations of these directions in latent space. This design combines the optimization flexibility of continuous attacks with the semantic realism of discrete rephrasing-based attacks. Experiments demonstrate that REALISTA achieves superior or comparable performance to state-of-the-art realistic attacks on open-source LLMs and, crucially, succeeds in attacking large reasoning models under free-form response settings, where prior realistic attacks fail.

Large language models (LLMs) can answer many questions impressively well, but changes in wording can sometimes cause them to make things up. These false or unreliable responses are often called hallucinations. We aim to evaluate whether a model remains reliable across meaning-preserving rephrasings of user questions, which commonly occur in real-world use. To study this, we developed a method called REALISTA. The idea is to create new versions of a user’s question that keep the same meaning but use different wording. Instead of changing the question randomly, REALISTA first finds natural, meaning-preserving edits and then combines them in the model’s internal representation space. This lets us explore many possible rephrasings while ensuring that the final question still means the same thing as the original one. We found that REALISTA can expose hallucinations in both open-source and commercial LLMs. This is important because a trustworthy model should not change from a correct answer to a false one just because a user phrases the same question differently. Our work provides a strong way to test whether LLMs remain reliable under realistic variations of user inputs.