Timezone: »

PareCO: Pareto-aware Channel Optimization for Slimmable Neural Networks
Ting-wu Chin

Fri Jul 17 08:20 AM -- 08:30 AM (PDT) @ None
Event URL: https://slideslive.com/38931679/pareco-paretoaware-channel-optimization-for-slimmable-neural-networks?ref=account-folder-55868-folders »

Slimmable neural networks have been proposed recently for resource-constrained settings such as mobile devices as they provide a flexible trade-off front between prediction error and computational cost (such as the number of floating-point operations or FLOPs) with the same storage cost as a single model. However, current slimmable neural networks use a single width-multiplier for all the layers to arrive at sub-networks with different performance profiles, which neglects that different layers affect the network's prediction accuracy differently and have different FLOP requirements. We formulate the problem of optimizing slimmable networks from a multi-objective optimization lens, which leads to a novel algorithm for optimizing both the shared weights and the width-multipliers for the sub-networks. While slimmable neural networks introduce the possibility of only maintaining a single model instead of many, our results make it more realistic to do so by improving their performance.

Author Information

Ting-wu Chin (Carnegie Mellon University)

More from the Same Authors

  • 2020 : Poster session »
    Janis Klaise · Lang Liu · Begum Taskazan · Lasse F. Wolff Anthony · Clive Cox · Omid Aramoon · Ting-wu Chin · Alexander Lavin