Newton-coupled Dual-Teacher Semi-supervised Learning Framework

Most semi-supervised learning frameworks rely on a single teacher that transfers zero-order supervision through pseudo-labels, constraining the student to imitate categorical outputs without perceiving the loss geometry. This design often leads to unstable optimization and limited generalization under scarce labels. We propose TTN (Two-Teachers Newton-guided Learning), a dual-teacher framework that integrates complementary supervision from MAE and DINOv3 and optimizes the student through a Newton step update. The two teachers provide multi-scale structural and semantic cues whose pseudo-labels and local Hessians are fused by confidence weighting, forming a unified second-order supervision signal. The student updates parameters preconditioned by the fused curvature, enabling stable convergence and geometry-consistent learning. TTN consistently improves over existing single-teacher and consistency-based semi-supervised learning methods on ImageNet, CIFAR-10, SVHN, and STL-10, demonstrating that combining multi-view self-supervised teachers with curvature-guided optimization yields robust and efficient semi-supervised learning.

This paper studies how to train image recognition models when only a small part of the data has human-provided labels. Many current methods use one model as a teacher to guess labels for the unlabeled images, but these guesses can be noisy and may make training unstable. We propose TTN, a method that uses two different teacher models instead of one. One teacher captures detailed visual structure, while the other captures broader visual meaning. TTN combines their guidance and uses it to train a student model more steadily. The main goal is to make learning from limited labeled data more reliable. Experiments on standard image recognition datasets show that TTN can improve accuracy and training stability compared with several existing methods. This may help reduce the need for costly manual labeling in visual machine learning tasks.