Frozen LLM Column Embeddings Are Strong Baselines for Column Type Annotation

Column type annotation (CTA), the task of assigning semantic labels such as Name, City, or Date to table columns, is central to automatic understanding of relational tables. Existing approaches often rely on handcrafted features, task-specific fine-tuning, or prompt-based inference. This paper studies a simpler alternative: using frozen large language models (LLMs) to produce column embeddings and training lightweight classifiers on top. Across six benchmark datasets and ten baselines, frozen LLM column embeddings outperform the strongest prior baseline by 3.7 points without any LLM fine-tuning. Results further show that much of the gain comes from the embedding layer itself, while classifier choice contributes only smaller additional improvements. We further analyze pooling strategies and LLM backbones to better understand the design choices behind frozen LLM-based column embeddings. Overall, strong CTA performance can be achieved without task-specific fine-tuning or prompt engineering, providing a practical baseline for column type annotation.