XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decoding

Current auto-regressive models can generate high-quality, topologically precise meshes; however, they necessitate thousands—or even tens of thousands—of next-token predictions during inference, resulting in substantial latency. We introduce XSpecMesh, a quality-preserving acceleration method for auto-regressive mesh generation models. XSpecMesh employs a lightweight, multi-head speculative decoding scheme to predict multiple tokens in parallel within a single forward pass, thereby accelerating inference. We further propose a verification and resampling strategy: the backbone model verifies each predicted token and resamples any tokens that do not meet the quality criteria. In addition, we propose a distillation strategy that trains the lightweight decoding heads by distilling from the backbone model, encouraging their prediction distributions to align and improving the success rate of speculative predictions. Extensive experiments demonstrate that our method achieves a $1.7\times$ speedup without sacrificing generation quality. Our code will be released.

Creating detailed 3D meshes with current AI models can produce high-quality results, but it is often slow because the model has to build the mesh step by step, making many small decisions one after another. This paper introduces XSpecMesh, a method that makes this process faster while keeping the generated meshes at the same quality level. The key idea is to let a small assistant component quickly suggest several upcoming parts of the mesh at once. The original model then checks these suggestions and only keeps the ones that are reliable. If a suggested part is not good enough, the model replaces it with a better one. We also train the assistant component to better match the behavior of the original model, so that its suggestions are more likely to be accepted. In experiments, XSpecMesh speeds up mesh generation by up to 1.7 times without reducing the quality of the generated results. This makes high-quality 3D mesh generation more efficient and practical.