pTNAS: Progressive Neural Architecture Search for Tabular Data

Recent advances have shifted the paradigm of tabular learning toward tabular foundation models, yet their accuracy relies on a heavy inference cost that scales poorly with context size. Deep neural networks remain a highly competitive and more efficient modeling paradigm when equipped with well-designed architectures; however, identifying such architectures in a data-adaptive and budget-aware manner remains challenging. We propose pTNAS, the first progressive neural architecture search (NAS) approach tailored for tabular data, which enables fast identification of a viable architecture and continuously improves its search performance as more budget becomes available. pTNAS adopts a filter-and-refine optimization strategy that combines efficient training-free and effective training-based architecture evaluation. In the filtering phase, we introduce pTProxy, a novel zero-cost proxy specifically designed for tabular networks that jointly captures architectural trainability and expressivity, enabling fast filtering of large architecture search spaces. In the refinement phase, pTNAS employs a fixed-budget scheduling algorithm to accurately identify the best-performing architecture from a small set of promising candidates. We further propose a budget-aware coordinator to optimize budget allocation holistically. Experiments show that pTNAS reduces the time to reach the globally best architecture by up to 82.75 X compared with other NAS approaches, achieves the best average predictive rank, and improves end-to-end efficiency by up to 4.78 X compared with TabPFN.

Many important decisions in finance, healthcare, online services, and business operations rely on data stored in tables. Recent large AI models for tabular data can make accurate predictions, but they can also be slow and expensive to use when many examples must be processed. This raises a practical question: can we build smaller models that are still accurate, but much faster to use? We introduce pTNAS, a method that automatically designs a suitable neural network for each tabular dataset. Instead of trying every possible design, pTNAS first uses a fast screening step to remove weak candidates, then spends more time testing only the most promising ones. This lets it find a useful model quickly and improve the model when more search time is available. Our experiments show that pTNAS can find strong tabular models much faster than existing automated design methods, while avoiding the high prediction cost of large tabular foundation models. This can make accurate machine learning on tabular data more practical for real applications with limited computing resources or strict response-time requirements.