Focusing: View-Consistent Sparse Voxels for Efficient 3D VAE Training

High-fidelity 3D generation depends on 3D VAEs that can compress and reconstruct complex geometry at high resolution. Recent methods often convert raw meshes into signed distance fields (SDFs), but this preprocessing can be lossy, especially for open or non-watertight assets. Render-supervised VAEs such as TripoSF avoid this conversion by matching rendered depth and normal maps, yet render losses supervise only the visible geometry for each view, leaving many latent voxels weakly constrained while still incurring unnecessary decoding and attention costs. We introduce Focusing, a view-consistent sparse-voxel training scheme for efficient 3D VAEs. Given a training view, Focusing performs depth-driven voxel carving directly in the structured latent space: voxels inconsistent with the rendered depth are removed before decoding, allowing the decoder and attention layers to operate only on locally relevant geometry. This view-dependent sparsity reduces memory and computation while concentrating learning on surface regions that contribute to the render. To stabilize training across shapes and viewpoints, we further propose adaptive zooming, which adjusts camera intrinsics to keep the number of active voxels within a target range and strengths supervision for fine details. The VAE is trained with render-based depth, normal, mask, and perceptual losses, together with sparse-voxel total variation and a brief TSDF warm-up to improve convergence and suppress holes. Across standard reconstruction benchmarks, Focusing improves Chamfer Distance and F-score over strong baselines while substantially reducing video random access memory (VRAM) consumption, enabling $1024^3$-resolution VAE training with as little as 50 GB of VRAM. These results demonstrate that local, view-consistent sparsity is an effective path toward higher-resolution and more efficient 3D VAE training.

Creating detailed 3D objects for games and virtual reality using AI typically requires large amounts of computing power. Current methods often waste time and memory processing invisible parts of an object, such as hidden interiors or empty surrounding space, when training models based on visible views. To address this, we developed Focusing, a method that concentrates computation only on the visible and important parts of a 3D object. During training, the system determines which regions will actually appear in the final view and removes hidden parts that do not affect the result. It also uses adaptive zooming to automatically adjust the view and preserve fine surface details without increasing memory demands. By avoiding unnecessary computation, Focusing greatly reduces the memory needed to train 3D content generation system, making the creation of realistic 3D assets more affordable and more accessible.