Link prediction, which is to predict the existence of a link/edge between two vertices in a graph, is a classical problem in machine learning. Intuitively, if it takes a long distance to walk from $u$ to $v$ along the existing edges, there should not be a link between them, and vice versa. This motivates us to explicitly combine the distance information with graph neural networks (GNNs) to improve link prediction. Calculating the distances between any two vertices (e.g., shortest path, expectation of random walk) in training is time consuming. To overcome this difficulty, we propose an anchor-based distance: First, we randomly select $K$ anchor vertices from the graph and then calculate the shortest distances of all vertices in the graph to them. The distance between vertices $u$ and $v$ is estimated as the average of their distances to the $K$ anchor vertices. After that, we feed the distance into the GNN module. Our method brings significant improvement for link prediction with few additional parameters. We achieved state-of-the-art result on the drug-drug-interaction (i.e., DDI) and protein-protein-association (i.e., PPA) tasks of OGB~\cite{hu2020ogb}. Our code is available at \url{https://github.com/lbn187/DLGNN}.