Incremental Transformer Neural Processes

Neural Processes (NPs), and specifically Transformer Neural Processes (TNPs), have demonstrated remarkable performance across tasks ranging from spatiotemporal forecasting to tabular data modelling. However, many of these applications are inherently sequential, involving continuous data streams such as real-time sensor readings or database updates. In such settings, models should support cheap, incremental updates rather than recomputing internal representations from scratch for every new observation—a capability existing TNP variants lack. Drawing inspiration from Large Language Models, we introduce the Incremental TNP ($\texttt{incTNP}$). By leveraging causal masking, Key-Value (KV) caching, and a data-efficient autoregressive training strategy, $\texttt{incTNP}$ matches the predictive performance of standard TNPs while reducing the computational cost of updates from quadratic to linear time complexity. We empirically evaluate our model on a range of synthetic and real-world tasks, including tabular regression and temperature prediction. Our results show that, surprisingly, $\texttt{incTNP}$ delivers performance comparable to—or better than—non-causal TNPs while unlocking orders-of-magnitude speedups for sequential inference. Finally, we assess the consistency of the model's updates---by adapting a metric of "implicit Bayesianness", we show that under a one-at-a-time streaming protocol, $\texttt{incTNP}$ retains a prediction rule as implicitly Bayesian as standard non-causal TNPs, demonstrating that $\texttt{incTNP}$ achieves the computational benefits of causal masking without sacrificing the consistency required for streaming inference.

AI systems are trained to make predictions that are correct for a collection of past observations, such as weather station readings or entries in a database. In real-world settings, these observations often arrive one at a time, but such systems usually have to reprocess the whole collection whenever a new observation appears. This makes frequent updating slow and expensive. We introduce Incremental Transformer Neural Processes, an uncertainty-aware prediction model that borrows an idea from modern language models: remembering useful information from previous observations so that only the new observation needs to be processed. Across synthetic and real-world tasks, including spreadsheet-style prediction problems and weather forecasting, our model updates much faster while matching or surpassing the accuracy of existing approaches. We also test whether this speed-up makes the model’s predictions less statistically consistent, and find that it behaves similarly to the standard model in the streaming setting we study. This could help make powerful prediction models more practical for live applications where data continually changes, such as environmental monitoring, forecasting, and evolving databases.