When Does a Low-Rank Bayesian Neural Network Certify Its Deterministic Center?

We study when a structured low-rank Gaussian variational posterior can certify a deterministic predictor in a Bayesian neural network with factorized layers $W_i=A_iB_i^\top$. The same low-rank Bayesian model gives rise to three natural certification targets: the posterior Gibbs predictor, the posterior predictive mean, and a deterministic center network. This paper focuses on the deterministic-center route. The main obstruction is factor non-identifiability: $(A_i,B_i)$ and $(cA_i,c^{-1}B_i)$ induce the same weight matrix but different factor norms, so a naive PAC-Bayes margin certificate in factor coordinates is representation dependent. We resolve this by passing to balanced factors obtained from the singular value decomposition of the center weights. On this balanced factor space, we combine Neyshabur's PAC-Bayes margin framework with rectangular Gaussian operator-norm bounds to derive explicit perturbation budgets and a margin bound for a deterministic center network. In the balanced Gaussian variational setting, we give sufficient conditions on the learned posterior scales under which the variational posterior itself serves as the certifying perturbation law, thereby yielding a PAC-Bayes margin bound for the corresponding deterministic center network through the actual posterior geometry rather than through an auxiliary perturbation chosen only for analysis. Under matched covariances, the resulting complexity term reduces to a sum of nuclear-norm contributions from the center weights, yielding rank-sensitive control and potentially sharper certificates when the intrinsic ranks are small.