Safety Game: Inference-Time Alignment of Black-Box LLMs via Constrained Optimization

Ensuring that large language models (LLMs) comply with safety requirements is a central challenge in AI deployment. Existing alignment approaches operate primarily during training, such as through fine-tuning or reinforcement learning from human feedback, but these methods are costly and inflexible, requiring retraining whenever new requirements arise. Recent efforts toward inference-time alignment mitigate some of these limitations but still assume access to model internals, which is impractical, and not suitable for third party stakeholders who do not have access to the models. In this work, we propose a model-independent, black-box framework for safety alignment that does not require retraining or access to the underlying LLM architecture. As a proof of concept, we address the problem of trading off between generating safe but uninformative answers versus helpful yet potentially risky ones. We formulate this dilemma as a two-player zero-sum game whose minimax equilibrium captures the optimal balance between safety and helpfulness. LLM agents operationalize this framework by leveraging a linear programming solver at inference time to compute equilibrium strategies. Our results demonstrate the feasibility of black-box safety alignment, offering a scalable and accessible pathway for stakeholders, including smaller organizations and entities in resource-constrained settings, to enforce safety across rapidly evolving LLM ecosystems.

Large language models are most useful when they are both safe and helpful, but balancing the two is hard: clamp down too much and the model refuses ordinary questions; loosen up and it can produce harmful content. Today, adjusting that balance usually means retraining or fine-tuning the model, or rewiring its internals which are expensive, and impossible if you reach the model only through a paid interface, as most smaller organizations do. We built a method that adds a safety layer on top of any model without changing it: no retraining, no fine-tuning, and no access to its inner workings, working entirely from the model's outputs as it runs. For each question it weighs a handful of possible answers and chooses among them as if playing a game against uncertainty: unable to tell whether a user's intent is innocent or harmful, it picks the most helpful answer it can while keeping overall risk under a chosen limit, and returns a safe default when nothing qualifies. A small, fast calculation makes the choice. This lets anyone set their own safety standards on top of any AI system, even without the resources to retrain large models. The improvement comes from how the method chooses among answers, not from judging each answer's risk more accurately: given the very same risk judgments, it keeps safe, useful answers that a simple pass/fail filter would wrongly reject.