Near-Universal Multiplicative Updates for Nonnegative Einsum Factorization

Despite the ubiquity of multiway data across scientific domains, there are few performant and user-friendly methods that fit non-standard nonnegative tensor factorization models tailored to the data at-hand. Researchers may use gradient-based automatic differentiation, which often struggles under nonnegative constraints, choose between a limited set of methods with mature implementations, or implement their own model from scratch. As an alternative, we introduce NNEinFact, an einsum-based multiplicative update algorithm that fits any nonnegative tensor factorization expressible as a tensor contraction by minimizing one of many user-specified loss functions, including the $(\alpha,\beta)$-divergence. To use NNEinFact, the researcher specifies their model with a string. NNEinFact converges to a stationary point of the loss, supports missing data, and fits to tensors with hundreds of millions of entries in seconds. Empirically, NNEinFact fits custom models which outperform standard ones in prediction tasks on real-world tensor data by over 37% and attains less than half the test loss of gradient-based methods while converging up to 90 times faster. Software is publicly available at github.com/jhood3/einfact.

Scientists regularly analyze massive datasets that track multiple interacting variables—such as gene expression levels across varying cell types, spatial coordinates, and time. Uncovering latent structures in this multiway data requires specialized tensor factorizations. Currently, researchers must either rely on generic models that fail to capture domain-specific interactions, or manually derive and implement complicated optimization algorithms from scratch. This paper automates this mathematical overhead. Instead of writing custom numerical solvers, a researcher simply uses a short text string to specify their scientific model. NNEinFact then automatically generates and runs the specific algorithm needed to perform statistical inference on the model of choice. The algorithm comes with mathematical guarantees, ensuring its rigor. Researchers can rapidly construct, test, and refine custom models tailored to their data without much technical expertise. In empirical evaluations, this paper’s method runs faster and achieves substantially lower error rates than standard methods, enabling accurate, interpretable structure discovery in large-scale scientific data.