Controllable Molecule Generation via Sparse Representation Editing: An Interpretability-Driven Perspective

Controllable molecule generation is crucial for diverse scientific applications, such as drug discovery and materials design. While large language models (LLMs) show great promise, their dense and entangled representations impede precise control over the generation of molecules with bespoke substructures or properties. To address this, we propose Sparse Representation Editing (SpaRE), an interpretability-driven framework for fine-grained and precise control in LLM-based molecule generation. The crux of SpaRE is to learn an overcomplete sparse feature space that disentangles LLM representations into a compact set of latent features corresponding to chemically meaningful concepts. Within this space, we can directly manipulate these concept-aligned latent features to achieve (1) local control, by generating target atoms and functional groups at specified positions; and (2) global control, by customizing the overall structural and physicochemical properties within defined ranges. In this way, our framework advances interpretability from post-hoc analysis to actionable generative control. Experiments show that SpaRE can generate chemically desirable molecules under complex constraints in real-world scenarios, while offering mechanistic insights for quantitative structure–property analysis. The code and demo are available at https://github.com/WanyuGroup/ICML2026_SpaRE.

Designing new molecules is key to finding better medicines and materials, but it is hard to tell today’s AI generators exactly what to make, such as placing a functional group at a specific site or increasing solubility without changing the main scaffold. Large language models can generate molecule strings well, yet their internal representations are tangled, so small changes often lead to unpredictable results. We introduce SpaRE, a method that learns a set of sparse, chemically meaningful concept controls inside the model from its own intermediate signals. Adjusting these controls steers generation in two ways. For local control, we can swap a specific atom or functional group at a chosen position. For global control, we can smoothly tune overall properties like solubility, aromaticity, or hydrogen bonding. Importantly, this control happens at inference time without retraining the language model. Across multiple benchmarks and real-world molecular design tasks, SpaRE generates valid, novel, and synthesizable molecules while meeting complex constraints with high success. Because the controls correspond to interpretable chemical concepts, SpaRE also helps explain why and how certain edits impact properties, supporting more reliable structure-property analysis.