Timezone: »

 
Spotlight
Sparsity in Partially Controllable Linear Systems
Yonathan Efroni · Sham Kakade · Akshay Krishnamurthy · Cyril Zhang

Wed Jul 20 02:50 PM -- 02:55 PM (PDT) @ Room 318 - 320
in Miscellaneous Aspects of Machine Learning/Reinforcement Learning »

A fundamental concept in control theory is that of controllability, where any system state can bereached through an appropriate choice of control inputs. Indeed, a large body of classical and modernapproaches are designed for controllable linear dynamical systems. However, in practice, we oftenencounter systems in which a large set of state variables evolve exogenously and independently of thecontrol inputs; such systems are only partially controllable. The focus of this work is on a large classof partially controllable linear dynamical systems, specified by an underlying sparsity pattern. Our mainresults establish structural conditions and finite-sample guarantees for learning to control such systems. Inparticular, our structural results characterize those state variables which are irrelevant for optimal control,an analysis which departs from classical control techniques. Our algorithmic results adapt techniquesfrom high-dimensional statistics—specifically soft-thresholding and semiparametric least-squares—toexploit the underlying sparsity pattern in order to obtain finite-sample guarantees that significantly improveover those based on certainty-equivalence. We also corroborate these theoretical improvements overcertainty-equivalent control through a simulation study.

Keywords: OPT: Control and Optimization · Reinforcement Learning · T: Reinforcement Learning and Planning

Author Information

Yonathan Efroni (Microsoft Research, New York)
Sham Kakade (Harvard University)
Akshay Krishnamurthy (Microsoft Research)
Cyril Zhang (Microsoft Research)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors