Cellular and tissue context is central to understanding health and disease, yet it is often inaccessible to machine learning analysis. In particular, protein interaction (PPI) networks have facilitated the discovery of disease mechanisms and candidate therapeutic targets; however, they are constructed by experiments in which much of the cellular and tissue context is removed. Given the advancements in single-cell sequencing technology, the limitations of homogeneous and static PPI networks are becoming more apparent. Here, we have developed a deep graph representation learning framework, AWARE, to inject cellular and tissue context into protein embeddings. AWARE optimizes for a multi-scale embedding space, whose structure reflects both intricate PPI connectivity patterns as well as cellular and tissue organization. We construct a multi-scale data representation of the Human Cell Atlas, and apply AWARE to learn protein, cell type, and tissue embeddings that uphold biological cell type and tissue hierarchies. We demonstrate the utility of such embeddings on the novel task of elucidating cell type specific disease-gene associations. For predicting the contributions of genes to diseases in different cell types, our AWARE protein embeddings outperform global PPI network embeddings by at least 12.5%, highlighting the importance of contextual embeddings for biomedicine.