DisjunctiveNet: Neural Symbolic Learning via Differentiable Convexified Optimization Layers

Many learning tasks in science and engineering are characterized by sparse datasets, which limits the effectiveness of purely data-driven approaches. At the same time, these problems are often accompanied by rich domain knowledge derived from physical laws, operational requirements, and expert heuristics. Such knowledge is frequently expressed as rules involving logical propositions and linear inequalities. Existing neuro-symbolic methods typically enforce these rules approximately through soft penalties, assume input-independent rules when designing specialized architectures, or rely on non-differentiable post-processing at inference time to achieve hard constraint satisfaction. While recent advances in differentiable optimization layers enable end-to-end feasibility enforcement within neural networks, extending these approaches to logical or mixed-integer rules remains challenging due to inherent nonconvexity. In this work, we propose a unified end-to-end framework for enforcing hard, input-dependent mixed integer linear constraints within neural networks. Our approach represents rules as disjunctive constraints and applies hierarchical convex relaxations to obtain convex hull formulations. These relaxations yield tractable linear constraints that can be embedded as differentiable optimization layers while enabling exact rule satisfaction. We demonstrate the effectiveness of the proposed framework on real-world datasets, achieving perfect rule satisfaction and strong predictive performance.

Many machine learning models make predictions based mainly on examples they have seen before. This works well when there is a lot of reliable training data, but it can be risky in science and engineering, where data are often limited and mistakes may violate known rules. For example, a building-control system may need to respect safety or operating limits, and a biological classifier may need to follow known relationships between marker genes and cell types. In these settings, experts often know useful rules in advance, but standard neural networks do not automatically obey them. This paper introduces a method for combining neural networks with such expert rules. The neural network first makes an ordinary prediction. Then, instead of accepting that prediction directly, our method checks which rules apply to the current input and adjusts the prediction to the closest one that satisfies those rules. This adjustment is done in a way that can still be included inside the training process, so the model can learn from data while also being guided by the rules. A key feature of the method is that it can handle rules with alternatives. For example, a rule may say that if a certain condition is true, then at least one of several possible requirements must hold. These kinds of “either-or” rules are common in practical decision-making, but they are difficult to enforce exactly in neural networks. Our approach rewrites these rules into a form that can be solved efficiently using standard optimization tools, while still guaranteeing that the final prediction satisfies the original rules when the rule system is feasible. We test the method on two tasks. The first is a cooling-control example, where a model predicts control actions such as fan speed, chiller usage, and pump power while satisfying operating and safety rules. The second is a single-cell biology classification task, where the model predicts cell types using gene-expression data and marker-gene rules. In both cases, the proposed method greatly improves rule satisfaction compared with ordinary neural networks and penalty-based methods that only encourage rule-following. The benefits are especially clear when the amount of training data is small or when the test data differ from the training data. The results suggest that machine learning models can become more reliable when they are trained not only from data, but also from trusted domain knowledge. This is particularly useful in scientific, engineering, and biomedical applications where predictions must be accurate and must obey known constraints. The method does add computational cost because it solves a small optimization problem for each prediction, but our experiments show that this cost is practical for the problem sizes studied.