XTransfer: Modality-Agnostic Few-Shot Model Transfer for Human Sensing at the Edge

Deep learning for human sensing on edge systems presents significant potential for smart applications. However, its training and development are hindered by the limited availability of sensor data and resource constraints of edge systems. While transferring pre-trained models to different sensing applications is promising, existing methods often require extensive sensor data and computational resources, resulting in high costs and limited transferability. In this paper, we propose XTransfer, a first-of-its-kind method enabling modality-agnostic, few-shot model transfer with resource-efficient design. XTransfer flexibly uses pre-trained models and transfers knowledge across different modalities by (i) model repairing that safely mitigates modality shift by adapting pre-trained layers with only few sensor data, and (ii) layer recombining that efficiently searches and recombines layers of interest from source models in a layer-wise manner to restructure models. We benchmark various baselines across diverse human sensing datasets spanning different modalities. The results show that XTransfer achieves state-of-the-art performance while significantly reducing the costs of sensor data collection, model training, and edge deployment.

What if building a strong human sensing system did not require collecting massive sensor datasets or training a new model from scratch? That is the key motivation behind XTransfer. In real-world human sensing, data are scarce, costly to label, privacy-sensitive, and often deployed on resource-constrained edge devices, making deep learning expensive and difficult to scale. XTransfer tackles this challenge in a simple yet powerful way. Instead of training new models with large amounts of data, it repairs and recombines parts of strong pre-trained models from other modalities, such as images or text, using only a few labeled sensor data. By adapting modality mismatch layer by layer and restructuring the model, XTransfer achieves state-of-the-art model performance while reducing the costs of data collection, training, and edge deployment. This makes practical human sensing systems much easier to build, deploy, and scale in the real world.