Efficient Data Selection for Split Neural Networks

Split Neural Networks (SplitNN) offer a great potential for distributed training of deep learning models across resource-constrained devices. However, severe computation and communication requirements restrict its practicality in scenarios with large number of participating clients and big-sized local datasets. While typical subset-selection techniques i.e., active learning and core-set selection, can potentially address these constraints, such approaches are impractical for SplitNN. In this paper, we propose a new framework for SplitNN to facilitate the existing subset-selection techniques. The proposed framework uses auxiliary networks with client-side models to generate pseudo-predictions on the local dataset and hence compute informative measures (e.g., entropy or least confidence) for subset-selection locally. Extensive experimental results show the effectiveness of the proposed framework, which substantially reduces computation and communication requirements while preserving the generalization performance.