Non-Parametric Optimization for Scalable Learning in Stochastic Decision Problems

Stochastic optimization (SO) plays a central role in decision-making under uncertainty. Among SO problems, time-varying stochastic optimization (TV-SO) is particularly important due to its applications in adaptive control and machine learning. Non-parametric approaches have been proposed for time-varying deterministic optimization; however, they have not been developed for their stochastic counterparts. This work addresses that gap by developing a stochastic variational framework based on Malliavin calculus. This framework yields non-parametric optimality conditions for SO problems with stochastic decisions and supports the design of a scalable deep-learning algorithm that is insensitive to the parameterization dimension. This algorithm, called the Stochastic Path Follower (SPF), is applied to two important problems under distribution drift, namely least-squares recovery and logistic regression. Experimental results show that the proposed approach outperforms state-of-the-art learning-based and gradient-based methods in both performance and scalability.

This paper provides the condition required to optimize time-varying stochastic optimization problems. Accordingly, it develops an efficient algorithm, called stochastic path follower (SPF), that it scalable to the neural network parameterization. This algorithm is competitive to the conventional pathwise differentiation method (PDM), however, SPF is more computationally effective as it requires less SDEs to be simulated for the problem solution. The developed algorithm is assessed on two important experiments within machine learning context, which show the effectiveness of the algorithm from performance and scalability viewpoints.