Single-cell multimodal omics technologies provide a holistic approach to study cellular decision making. Yet, learning from multimodal data is complicated because of missing and incomplete reference samples, nonoverlapping features and batch effects between datasets. To integrate and provide a unified view of multi-modal datasets, we propose Multigrate. Multigrate is a generative multi-view neural network to build multimodal reference atlases. In contrast to existing methods, Multigrate is not limited to specific paired assays while comparing favorably to existing data-specific methods on both integration and imputation tasks. We further show that Multigrate equipped with transfer learning enables mapping a query multimodal dataset into an existing reference atlas.