Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2× computational speedup, and dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.

Many machine learning methods seem fundamentally different: decision trees make a series of discrete choices, while diffusion models gradually transform noise into data through a continuous process. In this work, we show that these two approaches are actually describing the same underlying idea from different perspectives. We propose that both methods can be understood as learning how structure emerges through a step-by-step refinement process. In one case, this refinement appears as a hierarchy of decisions in a tree. In the other, it appears as a gradual denoising trajectory. Despite their differences, both can be viewed as learning a hidden path that explains how simple structure becomes complex data.To formalize this, we introduce Global Trajectory Score Matching (GTSM), a principle that captures the problem of learning the correct refinement process among many possible candidates. From this perspective, classic methods like gradient boosting can be interpreted as asymptotically optimal solutions. We demonstrate the usefulness of this viewpoint with two applications: Treeflow, a fast and high-quality generative model for tabular data, and DSMTree, a method for transferring decision-tree structure into neural networks with near-teacher performance. This work suggests that learning in these models is fundamentally about recovering structure through trajectories, rather than choosing between discrete or continuous representations.