IT$^3$: Idempotent Test-Time Training

Deep learning models often struggle when deployed in real-world settings due to distribution shifts between training and test data. While existing approaches like domain adaptation and test-time training (TTT) offer partial solutions, they typically require additional data or domain-specific auxiliary tasks. We present Idempotent Test-Time Training (IT3), a novel approach that enables on-the-fly adaptation to distribution shifts using only the current test instance, without any auxiliary task design. Our key insight is that enforcing idempotence---where repeated applications of a function yield the same result---can effectively replace domain-specific auxiliary tasks used in previous TTT methods. We theoretically connect idempotence to prediction confidence and demonstrate that minimizing the distance between successive applications of our model during inference leads to improved out-of-distribution performance. Extensive experiments across diverse domains (including image classification, aerodynamics prediction, and aerial segmentation) and architectures (MLPs, CNNs, GNNs) show that IT3 consistently outperforms existing approaches while being simpler and more widely applicable. Our results suggest that idempotence provides a universal principle for test-time adaptation that generalizes across domains and architectures.

Modern AI systems often fail when faced with unexpected changes in the data they are given — for example, when a camera sees an image in poor lighting or a sensor collects slightly different data than what the system was trained on. Traditionally, researchers have tried to fix this by designing special training tricks or feeding the system more examples. But these solutions don’t always work and often require extra data or task-specific adjustments.Our work introduces a simple and universal alternative called Idempotent Test-Time Training (IT³). Instead of crafting custom solutions, we teach the model a general principle: its predictions should stay the same if we run them through the model again. If they change too much, that’s a sign the input is unusual or corrupted. So during inference, we slightly update the model on-the-fly to make its predictions consistent — without needing new data or prior knowledge.This approach works well across very different tasks — from classifying images to predicting aerodynamic properties and mapping roads in aerial images — and helps models stay reliable even in unfamiliar conditions.