Hierarchical Policy Learning via Spectral Decomposition

In this paper, we identify a semantic decomposition in robot action sequences, separating task-level motion intent from execution-level refinements. By analyzing actions in the spectral domain using the discrete cosine transform (DCT), we observe that low-frequency components capture global motion trajectories, while high-frequency components encode precise timing, alignment, and contact behaviors. Motivated by this structure, we propose Causal Spectral Policy (CSP), which models action generation as a causal coarse-to-fine process: coarse motion is predicted from observation and language, and fine corrections are generated conditionally on the realized trajectory. Across simulation and real-world evaluations, CSP consistently outperforms strong baselines on precision-sensitive manipulation tasks. Additionally, we propose human-inspired teleoperation noise injection as a data augmentation method under which our approach demonstrates strong robustness to noisy demonstrations

Teaching robots to manipulate objects precisely is hard because good motion requires both a correct overall trajectory and tiny last-moment corrections, and these two things need to be learned differently. We show that converting robot movements into frequency components naturally separates these two levels. Our method, CSP, learns coarse motion first, then generates fine corrections conditioned on it. This improves performance on precision tasks and makes learning more robust to the noisy demonstrations that arise when humans operate robots remotely.