Non-Convex Learning with Guaranteed Convergence: Perspectives on Stochastic Optimal Control

We present and analyze a novel functional learning paradigm that operates on Reproducing Kernel Hilbert Spaces (RKHSs) without relying on the Representer Theorem, demonstrating its potential to learn stochastic optimal control policies in feedback form. Our methodology, based on the newly introduced concept of theé discrete derivative on RKHS, ensures that sequences generated by the iterative method remain within the intersection of all sublevel sets of the cost function, regardless of the chosen learning rate. In this way, we guarantee a consistent decrease in the cost function evaluation with each iteration until convergence, which is a significant finding in machine learning. By further decomposing the overall functional optimization problem into a suitable sequence of sub-problems, we create a cascade iterative method that computes each function in a domino effect. We briefly address the Witsenhausen counterexample problem, validating our convergence to a local minimum. Although the numerical solution obtained for the Witsenhausen counterexample problem is not yet on par with established ad-hoc methods, the preliminary results are promising and offer a fresh perspective in the field of stochastic optimal control.