Ramba: Selective State-Space Models for Relational Deep Learning

Relational Deep Learning aims to learn directly on multi-table databases, yet current methods face a fundamental tension: Transformers' quadratic complexity prohibits the large contexts relational data demands, while GNNs sacrifice global context for efficiency. We introduce Ramba, the first selective state-space model for relational databases. Our approach features two innovations: (1) Topology-Aware Linearization, which processes cells via global columnar serialization in O(L) complexity while recovering relational structure through sparse entity and foreign-key attention masks; and (2) Schema Dynamic Gating, which modulates SSM state transitions based on semantic alignment between the currently scanned attribute and the prediction target, enabling cross-table relevance filtering without relying on value distributions. Together, these enable Ramba to ingest vast relational contexts while selectively retaining semantically relevant information, a combination existing architectures cannot achieve. Experiments demonstrate state-of-the-art performance with linear scalability across diverse relational benchmarks.

Relational databases store much of the world’s structured information, such as e-commerce transactions, customer records, medical studies, and financial logs. However, applying machine learning to these databases usually requires experts to manually design features and combine information across many linked tables. This process is expensive and often loses important relational information. This paper introduces RAMBA, a new model designed to learn directly from multi-table relational databases. The key idea is to process large database contexts efficiently while still keeping track of which tables and columns are relevant for a prediction task. RAMBA uses a linear-time sequence model to handle long relational contexts and a schema-aware gating mechanism to filter useful information based on column and table meanings. Across multiple relational learning benchmarks, RAMBA achieves stronger prediction performance than existing transferable relational models while scaling more efficiently to long contexts. This suggests that models like RAMBA can reduce the need for manual feature engineering and make machine learning on relational databases more practical.