Differential equations play an important role in many different domains as they are used to describe the change in various real world systems. Previous works combined neural networks with differential equations to specify the dynamic or learn the solution. In this paper, we propose a general framework for modeling the solutions to ordinary and partial differential equations which relies on satisfying certain requirements so that the learned model always corresponds to the solution of the target equation. In particular, we propose novel flow models based on an efficient matrix exponential transformation to model ODE solutions. We extend this to stochastic differential equations and discuss suitable training strategies. Finally, we design models that are solutions to PDEs while respecting the initial and boundary conditions. Our models can be used in physics-informed learning, as well as to learn the mappings between the function spaces by defining a neural operator. Throughout the experiments, we demonstrate the benefits of using our method both in terms of predictive and computational performance.