Modeling dynamic relations between recording channels and the long-term dependencies are critical in multivariate time series. Recent approaches leverage graph neural networks to capture the direct first-order relationship between channels. While this is useful to capture co-occurrence patterns, they do not reveal indirect higher-order relationships governed by latent processes. For example, electricity consumption at consumer ends can follow similar temporal patterns, the simple correlation hides the facts that the patterns are driven by several unrecorded factors such as working activities over the day, the humidity, and the sunlight intensity – to name a few. To this end, we propose a dual message-passing recurrent neural system that disentangles the observed recording processes from the unobserved governing processes. The messages are passed in both the bottom-up and top-down manners: The bottom-up signals are aggregated to capture governing patterns, while the top-down messages augment the dynamics of low-level processes. Each process maintains its own memory of historical data, allowing process-specific long-term patterns to form. The governing process memories are jointly accessible to each other, and they collectively capture the governing dynamics of the entire system. Throughout extensive experiments on real-world time-series forecasting datasets, we prove the robustness and efficiency of our approach across different scenarios.