Post-Training Language Models for Crosslingual Consistency

Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an information-theoretic definition of crosslingual consistency as a divergence bound between a model’s response distribution and its round-trip pushforward across languages. We then introduce penalized consistency optimization (PCO), a post-training procedure that couples this divergence with a Kullback–Leibler penalty to a fixed reference language model. Because direct optimization of PCO requires expensive on-policy roll-outs, we propose a tractable surrogate, direct consistency optimization (DCO), which can be optimized off-policy. Across diverse language models and 26 languages, DCO significantly improves crosslingual consistency, outperforms existing methods, and enables targeted alignment of low-resource languages.

AI language models often give different answers to the same question depending on what language it is asked in, which is a problem when you want the system to behave reliably across many languages. This paper tackles that problem. We design a way to measure how inconsistent a model is across languages. Then they developed a training technique — called Direct Consistency Optimization (DCO) — that teaches the model to give more consistent answers regardless of the language used to ask a question. A key practical advantage of their approach is that it is computationally affordable to run, unlike a more theoretically ideal version they also describe. When tested across 26 languages and several different AI models, DCO made responses noticeably more consistent across languages, performing better than existing approaches. It also worked particularly well for languages that have less data available on the internet, which are typically harder for AI models to handle.