SPR-RAFT: Parameter-Efficient Regression-Aware Fine-Tuning for Biomedical LLM Regression

Biomedical regression tasks require predicting continuous targets from heterogeneous and unstructured evidence. While Large Language Models (LLMs) provide a robust interface for reasoning over mixed modalities, they are inherently limited by their discrete tokenization and cross-entropy objectives, which lack awareness of numerical proximity. To bridge this gap, we present \textbf{SPR-RAFT}, a parameter-efficient and regression-aware framework that adapts frozen LLMs for high-precision regression. SPR-RAFT introduces a dual-module architecture: a learnable soft prompt that conditions the LLM to route numerical reasoning into a specific latent state, and a lightweight regression head anchored on a dedicated readout token for numerical reasoning consolidation. Crucially, we align these two modalities via a hybrid objective that combines distribution-based text generation with representation-based robust regression. This ensures the model remains both semantically coherent and numerically calibrated. With only about 1.6M trainable parameters ($\sim$0.04\% of a 4B-parameter backbone), SPR-RAFT consistently outperforms prompting strategies, standard fine-tuning, and non-LLM baselines across diverse biomedical benchmarks, including clinical trial duration, biological age estimation, and molecular property prediction.

What is this paper about? Many real-world questions in biology and medicine — for example, how long will a clinical trial run, what is a person's biological age, or how soluble is a given molecule — require predicting a single number from messy, mixed-format evidence such as patient records, trial protocols, and chemical structures. Today's most powerful AI assistants, large language models (LLMs), are great at reading such heterogeneous text but surprisingly bad at producing precise numbers, because they were trained to predict the next word, not to know that '5.10' is much closer to '5.11' than to '510'. What did we do? We propose SPR-RAFT, a small adapter that teaches a frozen LLM to act as a calibrated number predictor. The adapter is tiny — about 1.6 megabytes, or 0.04 percent of a 4-billion-parameter LLM — yet it consistently beats both standard LLM fine-tuning and dedicated non-LLM models across clinical trial duration, biological age, and molecular property tasks. Why does it matter? Because the adapter is so small, the same large model can be cheaply specialized to many biomedical regression problems by hot-swapping different adapters, lowering the cost of deploying calibrated predictive AI in resource-limited research settings.