Error Propagation in Dynamic Programming: From Stochastic Control to American Option Pricing

This paper investigates theoretical and methodological foundations for stochastic optimal control (SOC) in discrete time. We start formulating the control problem in a general dynamic programming framework, introducing the mathematical structure needed for a detailed convergence analysis. The associate value function is estimated through a sequence of approximations combining nonparametric regression methods and Monte Carlo subsampling. The regression step is performed within reproducing kernel Hilbert spaces (RKHSs), exploiting the classical KRR algorithm, while Monte Carlo sampling methods are introduced to estimate the continuation value. To assess the accuracy of our value function estimator, we propose a natural error decomposition and rigorously control the resulting error terms at each time step. We then analyze how this error propagates backward in time-from maturity to the initial stage-a relatively underexplored aspect of the SOC literature. Finally, we illustrate how our analysis naturally applies to a key financial application: the pricing of American options.

Many real-world problems require making sequences of decisions under uncertainty, including applications in finance, robotics, and reinforcement learning. In these settings, decisions made today influence future outcomes, making the underlying optimization problems difficult to solve exactly, especially in high dimensions. In this work, we study discrete-time stochastic optimal control through the lens of machine learning and dynamic programming. We propose a framework that combines Monte Carlo simulation with kernel-based regression methods to approximate value functions backward in time. Our main contribution is a rigorous analysis of how approximation errors arise and propagate through the dynamic recursion. In particular, we decompose the total error into three separate components: learning error, simulation error, and propagation error. This modular structure makes the framework flexible, allowing different machine learning models or sampling strategies to be incorporated independently. As an application, we study the pricing of American-style financial options. Numerical experiments show that our approach achieves competitive accuracy while remaining computationally efficient in moderately high-dimensional problems.