Parameter-Free Encoders Remain Viable for RDB Foundation Models

Given a relational database (RDB) storing heterogeneous tabular information, how can we predict missing (or future) values in some target column of interest? As the space of potential targets is vast across enterprise settings, it is preferable to avoid learning a new model from scratch each time there is a new prediction task. Frozen foundation models based on RDB-specific encoders provide a viable solution, but ideal design remains an open question. On the one hand, it has recently been argued that certain parameter-free subgraph encoders combined with single-table foundation models can achieve near SOTA performance, with no RDB-specific pre-training required. Meanwhile, other contemporary studies advocate for parameterized encoders pre-trained to exploit observable labels for learning task-specific representations. To address this ambiguity, we rigorously analyze RDB encoder properties specifically when labels are present as inputs, proving key limitations on the efficacy of trainable encoder parameters. As empirical validation, we demonstrate that parameter-free encoders are capable of SOTA performance in regimes where our analysis is most relevant.