Workshop: Subset Selection in Machine Learning: From Theory to Applications

Statistical Measures For Defining Curriculum Scoring Function

Anirban Dasgupta · Vinu Sankar Sadasivan

[ Abstract ]
Sat 24 Jul 3:35 p.m. PDT — 3:40 p.m. PDT


Curriculum learning is a training strategy that sorts the training examples by some measure of their difficulty and gradually exposes them in that order to the learner to improve the network performance. In this work, we first propose and study a dynamic curriculum learning algorithm. Our dynamic curriculum algorithm greedily samples a subset of the training data whose gradients are directed towards the optimal weight. Motivated by our insights from the dynamic curriculum learning ordering and implicit curriculum ordering, we introduce a simple, practical curriculum learning strategy that uses statistical measures such as standard deviation and entropy values to score the difficulty of data points for real image classification tasks. Further, we also use our algorithms to discuss why curriculum learning is helpful.