Modularity in deep neural networks has provided scaling efficiencies leading to state of the art performance across multiple domains. A critical challenge for these networks is how to build and maintain a library of generalizable behavior modules. In this work, we propose a novel framework for building and maintaining a library of behavior primitives called Primitive Imitation for Control (PICO). Unlabeled demonstrations are automatically decomposed into existing or missing sub-behaviors which allows the framework to identify novel behaviors while not duplicating existing behaviors. We compared our results to several related approaches across two environments and achieve both better label accuracy and reconstruction accuracy as measured by action prediction mean squared error.