OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing

Tool-calling text-to-image (T2I) agents can plan and execute multi-step tool chains to accomplish complex generation and editing queries. However, this capability introduces a new safety attack surface: harmful outputs may arise from tool orchestration, where individually benign steps combine into unsafe results, making prompt-only jailbreak techniques insufficient. We present OrchJail, an orchestration-guided fuzzing framework for jailbreaking tool-calling T2I agents. Its core idea is to exploit high‑risk tool‑orchestration patterns: by learning from successful jailbreak tool-calling traces and their causal relationships to prompt wording, OrchJail directly guides the fuzzing search toward prompts that are more likely to trigger unsafe multi‑step tool behaviors, rather than relying on surface‑level textual perturbations. Extensive experiments demonstrate that OrchJail improves jailbreak effectiveness and efficiency across representative tool-calling T2I agents, achieving higher attack success rates, better image fidelity, and lower query costs, while remaining robust against common jailbreak defenses. Our work highlights tool orchestration as a critical, previously unexplored attack surface and provides a novel framework for uncovering safety risks in T2I agents.

Text-to-image AI systems can create pictures from written requests, and newer systems can also call several tools in sequence, such as first drawing a scene and then adding or editing objects. This makes them more useful, but it can also create new safety risks: a request that looks acceptable at each individual step may still lead to an unsafe final image. We study this risk in tool-using text-to-image AI systems. Our work introduces OrchJail, an automated testing method that looks for prompts that expose these multi-step safety failures. Instead of only changing the wording of a request, OrchJail learns from previous failures to understand which sequences of tool use are more likely to cause problems, and then uses that information to guide further testing. Across several representative systems, this approach finds unsafe behaviors more effectively and with fewer attempts than prior prompt-based testing methods. The goal of this work is to give developers and auditors a clearer way to find and fix weaknesses before such systems are deployed.