Scaling-Aware Adapter for Structure-Grounded LLM Reasoning

Large language models (LLMs) are enabling reasoning over 2D and 3D structures, yet existing methods remain modality-specific and typically compress structural inputs through sequence-based tokenization or fixed-length query connectors. Such architectures either omit the geometric grounding requisite for mitigating structural hallucinations, or impose inflexible modality fusion bottlenecks that concurrently over-compress and suboptimally allocate structural tokens, thereby impeding the realization of generalized all-atom reasoning. We introduce Cuttlefish, a unified multimodal LLM that grounds language reasoning in geometric cues while scaling modality tokens with structural complexity. First, Scaling-Aware Patching leverages an instruction-conditioned gating mechanism to generate variable-size patches over structural graphs, adaptively scaling the query token budget with structural complexity to mitigate fixed-length connector bottlenecks. Second, Geometry Grounding Adapter refines these adaptive tokens via cross-attention to modality embeddings and injects the resulting modality tokens into the LLM, exposing explicit geometric cues to reduce structural hallucination. Experiments across interdisciplinary all-atom benchmarks demonstrate that Cuttlefish achieves superior performance in heterogeneous structure-grounded reasoning. Code: github.com/zihao-jing/Cuttlefish.

Large language models are increasingly expected to reason beyond plain text, including over objects, systems, and structures with complex spatial or relational information. However, most existing methods still convert these inputs into fixed or simplified representations. This can discard important details, especially when the input size varies greatly, and may cause the model to generate explanations that are not grounded in the actual structure. This paper introduces Cuttlefish, a framework that helps language models reason over structured inputs more reliably. Instead of forcing every input into the same fixed token budget, Cuttlefish dynamically allocates more tokens to more complex structures and fewer tokens to simpler ones. It also injects explicit structural evidence into the language model, so the model’s responses are better tied to the input rather than inferred from text patterns alone. In simple terms, Cuttlefish gives language models a more flexible and evidence-grounded way to understand complex structures. This improves their ability to reason over variable-sized structured data and reduces unsupported or hallucinated explanations.