Abstract: We motivate the use of differentiable probabilistic programming techniques in order to account for the large model-space inherent to astrophysical $\gamma$-ray analyses. Targeting the longstanding Galactic Center $\gamma$-ray Excess (GCE) puzzle, we construct a differentiable forward model and likelihood that makes liberal use of GPU acceleration and vectorization in order to simultaneously account for a continuum of possible spatial morphologies consistent with the Excess emission in a fully probabilistic manner. Our setup allows for efficient inference over the large model space using variational methods. Beyond application to $\gamma$-ray data, a goal of this work is to showcase how differentiable probabilistic programming can be used as a tool to enable flexible analyses of astrophysical datasets.