Bridging Distribution Shift in Imitation Learning via Taylor Expansions

We propose Taylor Series Imitation Learning (TaSIL), a simple augmentation to standard behavior cloning losses in the context of continuous control. TaSIL penalizes deviations in the higher-order Taylor series terms betweenthe learned and expert policies. We show that experts satisfying a notion of incremental input-to-state stability are easy to learn, in the sense that a small TaSIL-augmented imitation loss over expert trajectories guarantees a small imitation loss over trajectories generated by the learned policy, and provide sample-complexity bounds for TaSIL that scale as $\tilde\calO(1/n)$ in the realizable setting, for $n$ the number of expert demonstrations. Finally, we compare experimentally standard Behavior Cloning, DART, and DAgger with TaSIL-loss-augmented variants. In all cases, we show significant improvement over baselines across a variety of MuJoCo tasks.