Color perception has long remained an intriguing topic in vision and cognitive science. It is a common practice to classify a person as either "color-normal" or "color-blind", and that there are a few prevalent types. However, empirical evidence has repeatedly suggested that at best, categories for colorblindness only serve as approximations to real manifestations of it. To better understanding individual-level color perception, we propose a color perception model that unifies existing theories for color-normal and color-blind people, which posits a low-dimensional structure in color space according to which any given user distinguishes colors. We design an algorithm to learn this low-dimensional structure from user queries, and prove statistical guarantees on its performance. Taking inspiration from these guarantees, we design a novel data collection paradigm based on perceptual adjustment queries (PAQs) that efficiently infers a user’s color distinguishability profile from a small number of cognitively lightweight responses. In a host of simulations, PAQs offer significant advantages over the de facto method of collecting comparison-based similarity queries.