Timezone: »
We propose a novel deep neural network that is both lightweight and effectively structured for model parallelization. Our network, which we name as SplitNet, automatically learns to split the network weights into either a set or a hierarchy of multiple groups that use disjoint sets of features, by learning both the class-to-group and feature-to-group assignment matrices along with the network weights. This produces a tree-structured network that involves no connection between branched subtrees of semantically disparate class groups. SplitNet thus greatly reduces the number of parameters and requires significantly less computations, and is also embarrassingly model parallelizable at test time, since the network evaluation for each subnetwork is completely independent except for the shared lower layer weights that can be duplicated over multiple processors. We validate our method with two deep network models (ResNet and AlexNet) on two different datasets (CIFAR-100 and ILSVRC 2012) for image classification, on which our method obtains networks with significantly reduced number of parameters while achieving comparable or superior classification accuracies over original full deep networks, and accelerated test speed with multiple GPUs.
Author Information
Juyong Kim (Seoul National University)
Yookoon Park (Seoul National University)
Gunhee Kim (Seoul National University)
Sung Ju Hwang (UNIST / AItrics)
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Poster: SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization »
Mon. Aug 7th 08:30 AM -- 12:00 PM Room Gallery #103
More from the Same Authors
-
2021 Poster: Unsupervised Representation Learning via Neural Activation Coding »
Yookoon Park · Sangho Lee · Gunhee Kim · David Blei -
2021 Oral: Unsupervised Representation Learning via Neural Activation Coding »
Yookoon Park · Sangho Lee · Gunhee Kim · David Blei -
2021 Poster: Unsupervised Skill Discovery with Bottleneck Option Learning »
Jaekyeom Kim · Seohong Park · Gunhee Kim -
2021 Spotlight: Unsupervised Skill Discovery with Bottleneck Option Learning »
Jaekyeom Kim · Seohong Park · Gunhee Kim -
2019 Poster: Variational Laplace Autoencoders »
Yookoon Park · Chris Kim · Gunhee Kim -
2019 Oral: Variational Laplace Autoencoders »
Yookoon Park · Chris Kim · Gunhee Kim -
2019 Poster: Curiosity-Bottleneck: Exploration By Distilling Task-Specific Novelty »
Youngjin Kim · Daniel Nam · Hyunwoo Kim · Ji-Hoon Kim · Gunhee Kim -
2019 Oral: Curiosity-Bottleneck: Exploration By Distilling Task-Specific Novelty »
Youngjin Kim · Daniel Nam · Hyunwoo Kim · Ji-Hoon Kim · Gunhee Kim -
2018 Poster: Video Prediction with Appearance and Motion Conditions »
Yunseok Jang · Gunhee Kim · Yale Song -
2018 Oral: Video Prediction with Appearance and Motion Conditions »
Yunseok Jang · Gunhee Kim · Yale Song -
2017 Poster: Combined Group and Exclusive Sparsity for Deep Neural Networks »
jaehong yoon · Sung Ju Hwang -
2017 Talk: Combined Group and Exclusive Sparsity for Deep Neural Networks »
jaehong yoon · Sung Ju Hwang