How Language Models Process Negation

We study how Large Language Models (LLMs) process negation mechanistically. First, we establish that even though open-weight models often provide wrong answers to questions involving negation, they do possess internal components that process negation correctly. Their poor accuracy is due to late-layer attention behavior that promotes simple shortcuts; ablating those attention modules greatly improves accuracy on negation-related questions. Second, we uncover how models process negation. We consider two hypotheses: models could use attention heads that attend to the phrase being negated and suppress related concepts, or they could directly construct a representation of the entire negative phrase (e.g., representing "not gas" as a vector that promotes liquids and solids). We apply a range of observational and causal interpretability techniques on Mistral-7B and Llama-3.1-8B to show that models implement both mechanisms, with the "constructive" mechanism being more prominent. Combined, our work deepens the understanding of LLMs' internals, highlighting construction-dominant computations and the coexistence of competing mechanisms within LLMs.

Negation (e.g. using the word "not") is everywhere in language, yet AI language models often stumble over it. Ask a model "An animal that is not an amphibian is a ___" and it may say "frog" anyway. Does this mean these models simply can't reason about negation? We opened up the models to find out (equivalent to opening up the human brain and looking at the neurons). The answer is surprising: they do contain internal machinery that handles negation correctly. The problem is a competing "shortcut" reflex in the model. When we disable that shortcut, we see that the model can reason about negation correctly. We further investigate how the model reasons about negation. We find that the model uses both construction and suppression. For example, the model constructs a representation of "not gas" as "solid" or "liquid", while also suppressing the concept of "gas". Our key discovery is that models explicitly construct a negated representation to handle negation and that there exists competing mechanisms within the model.