Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring

Large language models (LLMs) are becoming increasingly capable, but the mechanisms of their thinking and decision-making processes remain unclear. Chain-of-thoughts (CoTs) have been commonly utilized to externalize LLMs' thinking, but this strategy fails to accurately reflect LLMs' thinking process. Techniques based on LLMs' hidden representations provide an inner perspective to improve the monitorability of their latent thinking. However, previous methods only try to develop external modules instead of making LLMs themselves easier to monitor. In this paper, we propose a novel method, TELLME, improving the transparency of LLMs and helping monitors identify unsuitable and sensitive behaviors. Furthermore, we showcase the effectiveness of TELLME on detoxification tasks, where LLMs achieve consistent improvement among multimodal test sets, distinct architectures, and varying parameter scales. We further analyze TELLME's improvement on LLMs' generalization ability from both optimal transport theory and empirical perspectives.

As AI models become more powerful, it is increasingly important to ensure that they behave safely and avoid producing harmful content. One challenge is that unsafe behavior is often difficult to detect from inside the model before it appears in the final answer. The internal signals related to safe and harmful responses can be mixed together, making it hard for monitoring tools to tell whether the model is about to produce a safe refusal or a harmful response. We introduce TELLME, a method that makes these internal signals easier to separate. Instead of only adding an external safety filter, TELLME trains the model so that examples of similar behavior are represented more closely together, while safe and harmful behaviors are pushed farther apart. In simple terms, it helps the model organize its internal information in a way that is easier to inspect. This makes the model easier to monitor: safety tools can better recognize risky behavior from the model's internal representations before harmful text is generated. Across a wide range of language and vision-language models, we find that TELLME improves safety monitoring and reduces harmful outputs, while largely preserving the model's performance on general tasks such as question answering and reasoning.