Physics-informed neural networks (PINNs) are a promising method that have been recently proposed as a potential mesh-free alternative to conventional numerical methods for solving partial differential equations (PDEs) that are key to our model of the physical world. However, these problems typically lack ground truth, making selection of more accurate PINN models difficult, especially as PINN training involves processes such as hyper-parameter tuning, as is common in machine learning. This is exacerbated as PINNs need to balance multiple objectives, comprising the governing PDE, and associated boundary or initial conditions. Hence, a Pareto front-based model selection strategy is proposed to guide the selection of better performing models. In this strategy, an approximation to the Pareto set of solutions with minimal PINN loss is first constructed for different balances of loss weights. A loss weight located on the convex part of the Pareto front is then selected to rescale the training loss across all solutions. Across our experiments, this rescaling demonstrates a strong correlation between the rescaled PINN loss and mean squared error (MSE) relative to simulated ground truth, thereby illustrating the effectiveness of this proposed strategy for PINN model selection.