Language System: A Lightweight Ranking Framework for Language Models

Conventional research on large language models (LLMs) has primarily focused on refining output distributions, with less attention to the decoding process that transforms these distributions into final responses. Although recent work on inference-time scaling with reward models highlights the importance of decoding, such methods often incur high computational costs and limited applicability.In this paper, we revisit LLM decoding through the lens of recommender systems, conceptualizing the decoding process as analogous to the ranking stage in recommendation pipelines. From this perspective, both traditional decoding methods and reward models show clear limitations, including redundancy. To address this, we propose Language System, a lightweight framework that reranks candidate responses using features extracted by the base model.Experiments across diverse tasks demonstrate that Language System achieves performance comparable to large-scale reward models with <0.5M additional parameters, significantly reducing overhead during both training and inference. This highlights the efficiency and effectiveness of our approach in unlocking LLM capabilities.