QiMeng-LibBench: Benchmarking LLM Agents for Library-Scale Cross-Architecture Migration

Cross-architecture migration of high-performance libraries dictates ecosystem readiness on emerging hardware. The challenge is twofold: disentangling library-scale dependencies and performance-critical kernels with ISA-specific SIMD intrinsics, often trading migration speed for peak performance. While LLM-based agents offer a promising approach, are confined to function-level tasks or scalar code, failing to assess agents’ capabilities and limitations in realistic, library-scale migration. We present QM-LibBench, a benchmark for cross-architecture library-scale code migration, featuring 85 critical kernels from widely used libraries, including OpenCV, libjpeg, and NCNN. It supports comprehensive evaluations of compilability, correctness, and performance across major transitions: ARM→RISC-V, x86→ARM, and ARM→LoongArch. Evaluation of 12 SOTA agent-LLM combinations on QM-LibBench reveals that, due to the lack of library-level navigation and hardware-aware optimization, agents regress to superficial pattern matching, yielding only 20.88\% correctness and 0.83 speedup for libjpeg. Motivated by these findings, we further propose FSCM, a multi-agent framework incorporating hardware-aware global reconfiguration and performance optimization. FSCM improves OpenCV correctness to 71\%. The benchmark and code are available at https://github.com/WisdomJoy/QM-LibBench.

Modern software often needs to run on many different types of hardware, such as mobile processors, cloud servers, and emerging chip architectures like RISC-V. However, adapting high-performance library from one hardware platform to another is extremely difficult and time-consuming because engineers must rewrite low-level, hardware-specific optimizations by hand. Recent AI coding agents based on large language models (LLMs) have shown impressive abilities on programming tasks, raising the question of whether they can automate this migration process. To study this, we present QM-LibBench, the first benchmark designed to evaluate AI agents on real-world library migration across hardware architectures. Our benchmark includes widely used libraries such as OpenCV, libjpeg, and ncnn, together with tests for compilation, correctness, and runtime performance. Our experiments show that current AI agents still struggle significantly with these tasks. While they can often generate code that looks plausible, they frequently fail to preserve correctness or performance because they rely on superficial pattern matching rather than deeper architectural understanding. We hope QM-LibBench will help drive the development of more reliable AI systems for real-world software engineering and hardware adaptation.