Active Continual Learning with Metaplastic Binary Bayesian Neural Networks

Always-on edge systems must keep learning as conditions change under tight compute budgets and must detect unreliable predictions. Bayesian binary neural networks are attractive in this setting, but mean-field Bernoulli posteriors can saturate on long non-stationary streams, wiping out epistemic uncertainty and freezing plasticity. We propose BiMU, derived from a bounded-memory variational objective that balances stability, plasticity, and forgetting. BiMU combines a data term with controlled relaxation toward the prior and an uncertainty-dependent step size that prevents saturation and sustains informative uncertainty. This non-degenerate posterior enables fully online, buffer-free active querying via Monte Carlo disagreement, reducing label queries and backpropagation updates under imbalance. BiMU sustains learning and strong OOD detection on 1000-task Permuted-MNIST, and on OpenLORIS-Object achieves up to 32× label/update savings at matched accuracy under class imbalance and feature compression.

Robots, smart sensors, and embedded cameras need AI systems that can keep learning as their surroundings change, while using very little energy. Binary neural networks are attractive for this setting because they represent information using only two states, $+1$ and $−1$. Bayesian binary networks go a step further by also estimating how uncertain their predictions are. However, over long periods of learning, these networks can become too certain. Their internal probabilities collapse toward fixed decisions, making the network rigid and less able to adapt to new conditions. Here, we introduce BiMU, a training rule designed to keep binary Bayesian networks adaptable. BiMU combines new evidence with a controlled form of forgetting, so old information does not accumulate without limit. It also adjusts learning according to uncertainty: stable knowledge is protected, while uncertain or outdated knowledge remains easier to update. Because uncertainty remains informative, the system can ask for human labels only on examples that are likely to matter. In our experiments, this allowed the system to maintain strong performance while using up to $32\times$ fewer labels and weight updates. The result is a step toward reliable, long-lasting AI that can keep learning directly on energy-constrained devices.