Doubly Outlier-Robust Online Infinite Hidden Markov Model

We derive a robust update rule for the online infinite hidden Markov model (iHMM) for when the streaming data contains outliers and the model is misspecified. Leveraging recent advances in generalised Bayesian inference, we define robustness via the posterior influence function (PIF), and provide conditions under which the online iHMM has bounded PIF. Imposing robustness inevitably induces an adaptation lag for regime switching. Our method, which is called Batched Robust iHMM (BR-iHMM), balances adaptivity and robustness with two additional tunable parameters. Across limit order book data, hourly electricity demand, and a synthetic high-dimensional linear system, BR-iHMM reduces one-step-ahead forecasting error by up to 67% relative to competing online Bayesian methods. Together with theoretical guarantees of bounded PIF, our results highlight the practicality of our approach for both forecasting and interpretable online learning.

Imagine you're tracking a patient's heart rate over time. Most of the time it follows familiar patterns, resting, exercising, and sleeping, but occasionally a faulty sensor produces a wildly wrong reading. The challenge is teaching a computer to tell the difference between "this patient just entered a new state" and "this is just a broken sensor reading." This is the problem we set out to fix. Existing online learning methods are surprisingly easy to fool; a single extreme data point can simultaneously corrupt what the model has learned and convince it that a whole new regime has appeared out of nowhere. These are two separate failure modes; our method, BR-iHMM, treats both simultaneously. It down-weights suspicious data so they can't hijack the model's memory, and it waits for a short run of consistent evidence before committing to a regime change, akin to a doctor who doesn't diagnose a new condition from a single anomalous test result. We prove mathematically that both fixes are necessary; neither alone is enough. Tested on various data, our model cuts forecasting error by up to 67% over competing methods, while actually running faster.