From Tabular to Time Series: Can TabPFN Handle Mixed Data? A Study on PhysioNet

This paper introduces a novel adaptation of TabPFN for mixed tabular and time-series data, addressing a critical gap in foundation models for structured data. We propose cross-modal embedding layers and temporal attention mechanisms to enable seamless integration of static and sequential features while preserving TabPFN's fast inference capabilities. Evaluated on clinical (PhysioNet), retail (M5), and trajectory (UEA) benchmarks, our method achieves 4.7-13.4\% higher accuracy than modality-specific baselines (Chronos, TabForestPFN) and hybrids (TSMixer), with 10.9× faster inference. The model demonstrates particular strength in low-data regimes (82.4\% accuracy with just 100 samples) and robustness to distribution shifts (6.2\% accuracy drop vs. 14.7\% for TSMixer %under COVID-19 disruptions). Theoretical contributions include a synthetic pretraining protocol for mixed data and ablation studies validating our architectural choices. Results show consistent improvements across healthcare, finance, and motion analysis tasks, establishing a new state-of-the-art for unified structured data modeling.