MTNL: A Unified Modeling Perspective for Enhancing Tensor Network Learning

Over the years, the unsupervised and supervised learning research directions of tensor networks (TNs) have mainly developed in parallel. In this paper, we provide a view for their cooperative advancement through a novel mixed tensor network learning (MTNL) framework that unifies the two fields. Specifically, inspired by supervised TN learning tasks, multiple TNs are fused in a deep-network style in MTNL, enhancing the expressive power for the unsupervised TN learning tasks. We then develop a more flexible TN structure search prior with theoretical guarantees for learning multiple TN structures, aligning with trends in many supervised learning setups. More interestingly, by combining these components within a Bayesian framework, we show that MTNL induces a lightweight uncertainty quantification mechanism that is theoretically guaranteed by its connection to the dropout-based counterpart problem, making the mechanism a potential alternative for large-scale learning problems. Finally, we demonstrate the effectiveness of the MTNL framework on tensor recovery, parameter-efficient fine-tuning, and tensor regression experiments.

This work proposes a novel framework called Mixed Tensor Network Learning (MTNL), which unifies tensor network (TN) supervised and unsupervised learning within a single modeling paradigm. By integrating these two areas from a unified perspective, MTNL, as a new modeling paradigm, achieves stronger representation capacity, more flexible structure search capability, and efficient predictive uncertainty estimation. With enhanced expressive power, it offers a new perspective for TN representations and can be used to improve data representation performance, such as improving tensor recovery performance. Furthermore, its flexible structure learning capability ensures robust performance and generalizes well to supervised TN learning settings. Finally, the efficient predictive uncertainty estimation mechanism has the potential to ensure the trustworthiness of MTNL learning (TN learning) in a fast-response and resource-friendly manner.