AutoMat: Physics-Guided Agentic Reasoning for Solving Ill-Posed Inverse Microscopy Problems

Reconstructing atomistic crystal structures from a single noisy STEM projection is an ill-posed inverse problem: multiple lattices can explain similar contrast, and purely feed-forward models cannot verify physical validity. We present AutoMat, a failure-aware agentic controller that performs inference-time hypothesis search with closed-loop verification to convert Scanning Transmission Electron Microscopy (STEM) images into simulation-ready crystal structures and downstream properties. AutoMat composes perception and physics modules—pattern-adaptive denoising, physics-guided template retrieval as a state-dependent auxiliary branch, symmetry-constrained atomic reconstruction, and MLIP-based relaxation/validation—and triggers rollback-and-retry when verification fails. For systematic evaluation, we introduce STEM2Mat-Bench, a benchmark dataset containing 450+ annotated samples. Performance is assessed using lattice root-mean-square deviation (RMSD), formation energy mean absolute error (MAE), and structure matching accuracy. Results demonstrate that AutoMat outperforms existing approaches including SOTA models, specialized domain tools, and closed-source multimodal large models. This work establishes a direct pathway from microscopic characterization to atomic-scale modeling, addressing a fundamental challenge in materials science.

Modern microscopes can take images where individual atoms are visible, but turning these images into complete atomic structures is still difficult and usually requires expert manual work. This paper introduces AutoMat, an automated system that helps convert noisy microscope images of materials into atomic structure files that can be used for computer simulations. AutoMat combines image enhancement, structure reconstruction, and physics-based checking, so it can detect when a proposed structure is likely wrong and try again. We also introduce a benchmark dataset, STEM2Mat-Bench, to evaluate how well different methods perform on this task. Our results show that AutoMat is more accurate than existing general-purpose vision-language models and specialized microscopy tools, especially for difficult noisy images. The goal is to help researchers connect microscopy experiments with atomic-scale modeling more efficiently, while still keeping expert validation for challenging cases.