Protein Design with Agent Rosetta: A Case Study for Specialized Scientific Agents

Large language models (LLMs) are capable of emulating reasoning and using tools, creating opportunities for autonomous agents that execute complex scientific tasks. Protein design provides a natural testbed: although machine learning (ML) methods achieve strong results, these are largely restricted to canonical amino acids and narrow objectives, leaving unfilled need for a generalist tool for broad design pipelines. We introduce Agent Rosetta, an LLM agent paired with a structured environment for operating Rosetta, the leading physics-based heteropolymer design software, capable of modeling non-canonical building blocks and geometries. Agent Rosetta iteratively refines designs to achieve user-defined objectives, combining LLM reasoning with Rosetta's generality. We evaluate Agent Rosetta on design with canonical amino acids, matching specialized models and expert baselines, and with non-canonical residues---where ML approaches fail---achieving comparable performance. Critically, prompt engineering alone often fails to generate Rosetta actions, demonstrating that environment design is essential for integrating LLM agents with specialized software. Our results show that properly designed environments enable LLM agents to make scientific software accessible while matching specialized tools and human experts.

Designing new proteins is important for applications such as medicine, biotechnology, and materials science, but today’s AI systems are usually built for narrow tasks and often cannot handle many of the unusual chemical building blocks used in real-world protein engineering. At the same time, powerful scientific software for protein design exists, but it is difficult for non-experts to use effectively. We developed Agent Rosetta, an AI agent that combines a large language model with Rosetta, a widely used physics-based protein design platform. Instead of generating a protein design in a single step, the agent reasons through the problem, proposes design actions, evaluates the results, and iteratively improves the design to match a user’s goals. Unlike many existing AI protein design methods, our system can also work with non-standard amino acids and more complex molecular geometries. We found that Agent Rosetta performs competitively with specialized protein design tools and human experts, even in settings where existing machine learning approaches struggle. Our results also show that simply giving instructions to a language model is often not enough: carefully designing the software environment the agent interacts with is critical. Our work suggests that AI agents could make advanced scientific software more accessible and help automate complex research workflows.