Online Bayesian bipartite matching is a central problem in digital marketplaces and exchanges, including advertising, crowdsourcing, ridesharing, and kidney exchange. We introduce a graph neural network (GNN) approach that acts as a continuous approximation to the intractable optimal online algorithm, which selects actions (e.g., which nodes to match) by computing each action's value-to-go (VTG)---the expected weight of the matching if the algorithm takes that action, then acts optimally in the future. We train a GNN to estimate VTG and show empirically that our method returns high-weight matchings across a variety of tasks. Moreover, we identify a common family of graph distributions in spatial crowdsourcing applications, such as rideshare, under which VTG can be efficiently approximated by aggregating information locally within graphs. This structure matches the local behavior of GNNs, providing theoretical justification for our approach.