Arboreal Neural Network

Connectionist models and symbolic models have long embodied two divergent paradigms: the former excel at differentiable representation learning yet struggle with transparency, while the latter deliver explicit rule-based reasoning but resist gradient-based optimization. We introduce Arboreal Neural Networks (ArbNN), a neural-symbolic framework that unifies these paradigms both computationally and conceptually. At the design level, ArbNN departs fundamentally from prior neuralized-tree models through a depth-aware routing mechanism and a topology-informed softmax aggregation, which together enable one-shot full-path gradient propagation and consequently achieving rapid and well-conditioned optimization dynamics. At the conceptual level, ArbNN reveals that decision-tree branching and self-attention routing are two realizations of the same conditional computation primitive. We prove a structural isomorphism between a decision tree and a single-query attention head, enabling a differentiable architecture that faithfully preserves symbolic decision logic. The defining property of ArbNN is Bidirectional Fidelity, ensuring that the neural module can be compiled from—and losslessly decompiled back into—a symbolic tree, yielding both ordering consistency in ranking behavior and explicit, auditable interpretability via reconstructed if-else rules. ArbNN further supports GBDT-based initialization, allowing it to inherit strong inductive biases and integrate seamlessly with existing production workflows. Empirically, ArbNN achieves state-of-the-art performance on various public tabular benchmarks and delivers consistent gains under temporal distribution shift in large-scale industrial credit-risk systems. To support realistic evaluation, we additionally construct TabCredit, a feature-rich, temporally partitioned dataset built from millions of real-world loan applications. Together, these results demonstrate that ArbNN forms a unified, reversible, and practically deployable bridge between symbolic reasoning and neural computation for high-stakes tabular domains.

ArbNN is motivated by the fundamental gap between decision trees and neural networks for tabular learning. Tree-based models such as XGBoost dominate high-stakes applications due to their interpretability and robustness, while neural networks deliver stronger end-to-end optimization but lack auditability. Existing neuralized-tree methods typically trade off interpretability and optimization quality. To address this, we propose Arboreal Neural Networks (ArbNN), a neural-symbolic framework that converts decision trees into our newly proposed differentiable neural ArborCells while preserving their original decision structure. The sparse routing within each ArborCell enables efficient full-path gradient optimization and post-hoc conversion into human-readable rules. We further establish a structural correspondence between tree routing and self-attention, providing a unified view of symbolic reasoning and neural routing. Experiments on public tabular benchmarks and a large-scale industrial credit-risk dataset show that ArbNN outperforms strong GBDT and deep learning baselines, especially under temporal distribution shift. Unlike black-box neural models, ArbNN maintains interpretability and reliable probability calibration for real-world decision systems.