Learning to Offload with Low Regret under Asymmetric Misclassification Costs

We consider edge AI systems designed for binary classification via hierarchical inference(HI): a compact local model on a resource-constrained device classifies each incoming sample, and when uncertain, forwards it to a more powerful model on a remote server over a costly wireless link, a decision called offloading. The device receives a stream of periodically generated samples and must classify each one in real time, deciding online whether to act locally or offload, trading offloading cost against misclassification risk. The challenge is that neither the local model's true accuracy at each confidence level nor the offloading cost is known in advance and must be learned from experience. We focus on the setting where the two types of misclassifications carry different costs and propose an offloading algorithm for which we prove the first $O(\log T)$ regret guarantee for asymmetric-cost hierarchical inference. We supplement our analytical results via experiments on four real-world datasets.