Scale-Invariant Empirical-Bayes Laplace Approximation for ReLU Networks

A post-hoc Bayesian procedure should be invariant to function-preserving reparameterizations. Standard empirical-Bayes Laplace violates this principle in ReLU networks. Positive homogeneity creates scale-equivalent parameterizations, but the isotropic evidence objective depends on the chosen representative through the prior norm and curvature log-determinant. We introduce $\gamma$-canonicalization, which fixes a canonical representative on each ReLU scale orbit, fits an isotropic Laplace approximation there, and transports the prior back to the original coordinates. The resulting precision field transforms compatibly with curvature, making the generalized evidence and Jacobian-linearized predictive distribution invariant for every $\gamma\in[0,1]$. Selecting $\gamma$ and the prior precision jointly by marginal likelihood yields an EB-Laplace procedure defined on ReLU scale-equivalence classes. Empirically, standard EB-Laplace varies across functionally identical rescalings, while $\gamma$-canonicalized Laplace removes this variation to zero.