GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data

We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones. We introduce Graph-guided Ordering with Local Refinement (GO-LR), show its equivalence to weighted Minimum Linear Arrangement, and interpret the practical solver as a TSP-path-style surrogate. We propose GOTabPFN,which builds on GO-LR, and a Neuro-Inspired Subunit Compression (NSC) unit to pool locally adjacent ordered features into meta-features, yielding a compact representation that makes TabPFN-style prediction practical in HDLSS regimes. Across tabular benchmarks, GOTabPFN improves stability and accuracy under tight token budgets.

Many real-world datasets, such as biomedical datasets, have thousands of measured variables but only a small number of samples. This makes prediction difficult because machine learning models can struggle to find reliable patterns from limited data. We introduce GOTabPFN, a method that helps existing tabular prediction techniques to work better in this setting without needing to retrain or to change their underlying model. GOTabPFN first arranges related variables near each other, then compresses nearby variables into a smaller set of informative summaries. This compact version of the data makes prediction more practical when the number of variables is much larger than the number of samples. Across several tabular datasets, GOTabPFN showed significant improvement in accuracy and stability.