State space models (SSMs) have shown great potential in sequence modeling across various ap- plications. Current initialization schemes for the SSM state matrix primarily rely on the HiPPO framework that is built on function approximation. In this paper, we aim to understand the role of the state matrix initialization in a different angle which can be used to explain the ben- efit of complex-valued SSMs. Furthermore, for complex-valued initialization, we uncover an approximation-estimation tradeoff when training SSMs with a specific class of target functions.