Efficient LLM Moderation with Multi-Layer Latent Prototypes

Although modern LLMs are aligned with human values during post-training, robust moderation remains essential to prevent harmful outputs at deployment time. Existing approaches suffer from performance-efficiency trade-offs and are difficult to customize to user-specific requirements. Motivated by this gap, we introduce Multi-Layer Prototype Moderator (MLPM), a lightweight and highly customizable input moderation tool. We propose leveraging prototypes of intermediate representations across multiple layers to improve moderation quality while maintaining high efficiency. By design, our method adds negligible overhead to the generation pipeline and can be seamlessly applied to any model. MLPM achieves state-of-the-art performance on diverse moderation benchmarks and demonstrates strong scalability across model families of various sizes. Moreover, we show that it integrates smoothly into end-to-end moderation pipelines and further improves response safety when combined with output moderation techniques. Overall, our work provides a practical and adaptable solution for safe, robust, and efficient LLM deployment.

AI language models can be manipulated by carefully crafted inputs, known as adversarial prompts, that trick them into producing harmful outputs. Defending against these attacks typically means choosing between two imperfect options: powerful safety filters that are slow and costly to run, or faster methods that sacrifice accuracy. We developed a new approach, called the Multi-Layer Prototype Moderator (MLPM), that sidesteps this trade-off. Rather than adding a separate safety system on top of a model, MLPM looks inside the model itself as it processes an incoming prompt. It examines signals from the model's internal layers and assesses how closely a prompt resembles patterns from known safe or unsafe examples, using a Mahalanobis distance. A built-in selection mechanism automatically identifies which internal layers carry the most useful information for a given model, ignoring the rest. The result is a safety detector that matches or surpasses the best existing methods across a range of standard benchmarks, while adding virtually no extra computational cost (less than 0.001% overhead). It also trains efficiently, reaching strong performance with as few as 1,000 examples.