Training Diffusion Language Models for Black-Box Optimization

We study offline black-box optimization (BBO), aiming to discover improved designs from an offline dataset of designs and labels, a problem common in robotics, DNA, and materials science with limited labeled samples. While recent work applies autoregressive LLMs to BBO by formatting tasks as natural-language prompts, their left-to-right design generation struggles to capture the strong bidirectional dependencies inherent in design problems. To address this, we propose adapting diffusion LLMs to offline BBO to leverage their bidirectional modeling capabilities. However, a domain gap exists between the natural text pre-training of diffusion LLMs and the heterogeneous signals in BBO (prompts, designs, and labels). To bridge this gap, we construct a unified prompt–-response corpus and introduce delimiter tokens to explicitly mark field boundaries for domain adaptation. We further propose a two-stage post-training framework to align the diffusion LLM generation with high-label designs. The first stage performs supervised fine-tuning on the unified dataset via masked-response prediction, and the second stage adopts reinforcement learning with rewards defined by label improvements. Our method achieves state-of-the-art results on Design-Bench under small-data settings. Code for our work is available here: https://github.com/zpointS/DiBO.

In scientific discovery fields like robotics and drug discovery, researchers need to find optimal designs — such as a robot body shape that runs fastest, or a DNA sequence that binds a protein most strongly — but testing each candidate through physical experiments is slow and expensive. Can a large language model help propose better designs by learning from existing data? We show that it can. We adapt a language model to act as a scientific design assistant: given existing robot designs and their speeds, prompted with "design a robot that runs faster," the model might reply "set the limb length to 0.45m and thigh angle to 35°"; given DNA sequences and their binding strengths, prompted with "design a DNA sequence that binds protein better," it might reply "try GCGTACGA for stronger binding." We use a diffusion language model rather than an autoregressive one because scientific designs exhibit bidirectional dependencies — unlike natural language's left-to-right nature. We then train it through a three-stage pipeline that progressively teaches it to generate high-performing designs. Our approach achieves state-of-the-art results across multiple scientific design tasks, with the entire training process completing in just 1.5 hours on a single GPU.