FRAGILE: Benchmarking Framing Sensitivity in High-Stakes Decision-Making

Large language models (LLMs) are increasingly deployed in high-stakes decision-making settings such as legal reasoning, where consistency under factually equivalent inputs is critical. However, we find that semantically equivalent but differently framed inputs can significantly destabilize LLM decisions, even when all underlying facts are preserved. To systemically investigate this problem, we introduce FRAGILE, a large-scale benchmark spanning moral reasoning, medical triage, legal judgment, and role conflict that isolates fact-preserving semantic framing across three controlled dimensions: temporal slice, value-tinted narration, and narrative vividness. Our experiments reveal a high susceptibility to framing, with an average decision flip rate of 28.6% across diverse architectures. These flips consistently follow the framing’s intended direction, and internal representations at the decision token reflect concepts aligned with the applied frame—confirming that framing-induced context, rather than factual content alone, governs LLM decisions. Given this contextual dependency, we evaluate whether explicitly anchoring decisions to values at the prompt level can mitigate such sensitivity. We find that prompt-based value anchoring fails to reliably suppress framing effects, indicating that the governing mechanism resides deeper than the prompt surface. Consequently, effective mitigation necessitates representation-level alignment that targets the specific contextual pathways activated by each framing type.