Graph models allow us to capture the hidden dependencies of the multivariate time-series when the graphs are constructed in a similarly dynamic manner. Previous dynamic graph models require a pre-defined and/or static graph structure, unknown in most cases, or they only capture the spatial relations between the features. In healthcare, the interpretability of the model is an essential requirement to build trust with clinicians. Here, we propose DynaGraph, an end-to-end interpretable contrastive graph model that learns the dynamic graph construction of the multivariate time-series as part of optimisation. We validate our model in two real-world clinical datasets at challenging imbalanced multi-label tasks. Compared to state-of-the-art dynamic graph models, DynaGraph achieves an improvement in balanced accuracy of 5.15% and 0.56%, and in sensitivity of 15.35% and 4.04% across the two datasets. Through a pseudo-attention approach to the graph construction, the model also provides time-resolved interpretability for clinical feature importance.