Copula-SVI: Vine-Copula Variational Inference with Stein Refining for Instance-Level Correlation Capturing

Mean-field variational inference (VI) is scalable, but its independence assumption can severely limit inference when the posterior is inherently coupled across instances especially for correlated data. Existing structured VI approaches either impose simple dependence patterns or incur substantial cost as dependence becomes richer, leaving efficient higher-order instance-level dependence modeling largely unresolved. We propose Copula-SVI, which augments amortized marginals with an explicit vine-copula posterior and refines joint samples with Stein updates toward the true posterior. The vine construction makes dependence learning and sampling practical by decomposing it into bivariate copula factors, enabling edge-minibatched training with variance-aware level-wise sampling and efficient dependence-aware initialization via a sparse vine built from the same sampled edges. Experiments on constrained clustering and time series modeling show consistent improvements over strong structured VI baselines and demonstrate efficient higher-order instance-level dependence modeling.

Many machine learning systems make predictions by estimating hidden uncertainty behind the data. A common way to do this is efficient, but it often treats different data points as independent. This can be a poor fit for correlated data, such as time series or groups of samples that influence one another. Our work studies how to model these relationships more effectively without making the computation too expensive. We introduce Copula-SVI, a method that first learns the uncertainty of each data point separately, and then connects these uncertainties through a flexible dependency model. To make this practical, we break a complicated web of relationships into many simpler pairwise relationships, similar to understanding a social network by looking at connections between pairs of people. We also refine the generated samples so that they better match the true hidden structure of the data. Experiments on clustering and time-series tasks show that modeling relationships between data points can lead to better inference. This makes uncertainty estimation more useful for real-world data where samples are rarely truly independent.