CAffNet: Hard Constraint-Affine Neural Networks

We present a novel framework for embedding hard constraint satisfaction into neural network (NN) architectures, specifically feedforward neural networks and transformers, with input-dependent affine constraints of arbitrary cardinality. Traditional constraint enforcement approaches either rely on penalty-based soft constraints, which offer no guarantee of satisfaction, or on post-processing methods that enforce constraints after the NN is trained, which may lead to suboptimality. We introduce a trainable constraint-affine (CAffine) layer into NNs, yielding CAffNet, which goes beyond enforcing affine constraints via fixed orthogonal or parallel projections and enables joint optimization with network parameters. Moreover, we impose no restrictions on the constraint space dimensions and establish that our construction preserves the universal approximation properties of NNs, while providing provable guarantees on constraint adherence for all inputs. Experimental validation demonstrates robust performance across diverse domains requiring guaranteed constraint satisfaction.

Neural networks are increasingly used in applications where their outputs must always satisfy certain rules, such as safety limits, resource constraints, or physical requirements. However, standard neural networks do not naturally guarantee that these rules will be followed. Existing methods often either add penalties during training, which cannot ensure perfect satisfaction, or correct the network’s output after training, which may reduce performance. In this work, we introduce CAffNet, a neural network framework that builds hard constraint satisfaction directly into the model. CAffNet adds a trainable constraint-affine layer that ensures the network output satisfies input-dependent affine constraints, meaning the required rules can change with the input. This layer is trained jointly with the rest of the network, allowing the model to learn effective solutions while still guaranteeing that the constraints are satisfied. We prove that CAffNet preserves the expressive power of neural networks and provides guaranteed constraint satisfaction for all inputs. Experiments across several tasks show that CAffNet performs well while reliably enforcing the required constraints.