Profiling the Irrational Agent: Cognitive Modeling of LLM Behaviors in Sequential Jailbreaks

Large language models (LLMs) are increasingly deployed in high-stakes settings, yet they remain vulnerable to sequential jailbreaks that exploit multi-turn interaction to circumvent safety mechanisms. Current safety evaluations are largely outcome-based, offering little insight into the latent decision processes that lead to unsafe compliance. We propose an interpretable cognitive modeling framework that couples a controlled elicitation paradigm, the Contextual Iowa Gambling Task (C-IGT), with a Generalized Rescorla--Wagner (GRW) architecture to decompose behavior into measurable mechanisms. Across a diverse set of mainstream LLMs, we find that sequential vulnerability is not explained by scale alone but emerges from interactions among cognitive factors, including optimism-biased learning, perceptual reward amplification, and choice inertia. Moreover, counterfactual feedback and psychologically framed rewards (e.g., regret, authority, threat) substantially accelerate the transition from refusal to compliance. These results yield principled cognitive profiles of LLM "irrationality" and provide insights for interdisciplinary research on LLM agents at the intersection of machine learning and human behavioral science.

Large language models are increasingly used in real-world applications, but they can still be manipulated through multi-turn conversations. In these attacks, a user does not ask a harmful question directly in one step. Instead, they gradually pressure the model over several turns until it stops refusing and begins to comply. This paper studies why such failures happen. We propose a controlled way to test language models as decision-makers in repeated interactions. Instead of only recording whether an attack succeeds, our method examines how a model’s behavior changes over time. We measure patterns such as whether the model gives too much weight to positive feedback, keeps repeating its previous choices, overreacts to emotional or authoritative language, or becomes more likely to comply after seeing what it “missed” by refusing. Our experiments across many popular language models show that vulnerability is not simply determined by model size or reasoning ability. Some strong models can still be highly sensitive to repeated pressure, especially when the interaction uses regret, threat, or authority-based framing. We also find that simply increasing numerical rewards is less effective than using psychologically persuasive feedback. These findings suggest that safer AI systems require more than measuring final success or failure rates. Developers should also diagnose the behavioral patterns that lead models from safe refusal to unsafe compliance. Such behavioral profiles can help design more targeted safety evaluations and defenses for multi-turn interactions.