Mathematics Colloquium
Serdar Yuksel
Queen's University
Title: Stochastic Kernel Topologies and Implications on Approximations, Robustness, and Learning
Date: Friday, October 10th
Place and Time: Love 101, 3:05-3:55 pm
Abstract. Stochastic kernels represent stochastic processes, controlled and control-free system models, measurement channels, and control policies, and thus offer a very general mathematical framework in applied mathematics. We will first present several properties of such kernels and study several kernel topologies. These include weak* (also called Borkar) topology, Young topology, kernel mean embedding topologies, and strong convergence topologies. After a general introduction, we then study convergence, continuity, and robustness properties involving models and policies viewed as kernels, in the context of stochastic control but also related areas in applied mathematics. On models viewed as stochastic kernels; we study robustness to model perturbations, including finite approximations for discrete-time models and robustness to more general modelling errors and study the mismatch loss of optimal control policies designed for incorrect models applied to a true system, as the incorrect model approaches the true model under a variety of kernel convergence criteria: We show that the expected induced cost is robust under continuous weak convergence of transition kernels. Under stronger Wasserstein or total variation regularity, a modulus of continuity is also applicable. As applications of robustness under continuous weak convergence via data-driven model learning, (i) robustness to empirical model learning for discounted and average cost criteria is obtained with sample complexity bounds; and (ii) convergence and near optimality of a quantized Q-learning algorithm for MDPs with standard Borel spaces, which we show to be converging to an optimal solution of an approximate model, is established. In the context of continuous-time models, we obtain counterparts where we show continuity of cost in policy under Young and Borkar topologies, and robustness of optimal cost in models including discrete-time approximations for finite horizon and infinite-horizon cost criteria. Discrete-time approximations under several information structures will then be obtained via a unified approach of policy and model convergence. A concluding message is that weak kernel topologies are appropriate for policy spaces, and strong kernel topologies are suitable for studying models towards establishing very general existence, approximations, robustness and learning results. [Joint work with Ali D. Kara, Somnath Pradhan, Naci Saldi, Omar Mrani-Zentar].