TFM-Retouche: A Lightweight Input-Space Adapter for Tabular Foundation Models

Tabular foundation models (TFMs) such as TabPFN-2.6, TabICLv2, ConTextTab, Mitra, LimiX, and TabDPT achieve strong zero-shot performance through in-context learning, but their inductive biases remain fixed at inference time. Adapting a pretrained TFM to a specific dataset typically requires either full fine-tuning, which is expensive, or parameter-efficient methods such as LoRA, which must be tailored to the internal architecture of each TFM, with mixed empirical evidence on accuracy and calibration (Tanna et al., 2026; Rubachev et al., 2025). We introduce TFM-Retouche, a lightweight input-space residual adapter that is architecture-agnostic with respect to the frozen TFM backbone. The adapter learns a small residual correction in the input space to align the input data with the inductive biases of the pretrained model, and is trained end-to-end through the frozen TFM with a post-training identity guard that falls back to the unmodified TFM whenever adaptation does not help on held-out validation. On TabArena-Lite (Erickson et al., 2025) (51 datasets spanning binary classification, multiclass, and regression), TabICLv2-Retouche, the framework instantiated on TabICLv2, is the top-ranked method on the leaderboard with light per-task tuning and ensembling, lifting aggregate Elo by +56 over the frozen TabICLv2 base and sitting on the Pareto front of predictive quality versus both training and inference time.