DynaTok: Token-Based 4D Reconstruction from Partial Point Clouds

We address 4D reconstruction from partial point cloud sequences, where depth-sensor observations are incomplete, unordered, and lack explicit temporal correspondences. This geometry-only setting is challenging due to missing observations and ambiguous dynamics. While recent progress has largely relied on image-based methods, existing point-based approaches typically focus on single objects, assume relatively complete inputs, or require explicit correspondences. To address these limitations, we propose DynaTok, a point-based framework for correspondence-free 4D reconstruction from partial point cloud sequences without images. DynaTok encodes frames into compact latent tokens, aggregates incomplete observations over time with a Transformer-based spatiotemporal encoder, and decouples geometry and motion through residual tokens in a unified model. A flow-matching decoder then reconstructs complete, temporally consistent 4D point-cloud sequences conditioned on the latent tokens. Experiments on object- and scene-level benchmarks demonstrate improved reconstruction quality and temporal coherence from partial point cloud observations.

Many robots, AR systems, and scene-understanding applications need to understand how 3D environments change over time. However, depth sensors often capture only partial point clouds: parts of the scene may be missing, points are unordered, and the same physical point is not tracked across frames. This makes it difficult to reconstruct a complete and temporally consistent 4D scene, especially when objects move or are temporarily hidden. We propose DynaTok, a method that reconstructs dynamic 4D point-cloud sequences from these incomplete observations without using images or explicit point correspondences. DynaTok converts each frame into compact latent tokens, aggregates information across time with a spatiotemporal Transformer, separates persistent geometry from motion using residual tokens, and uses a flow-matching decoder to generate complete 4D point clouds. This research matters because it enables geometry-only dynamic scene reconstruction from realistic depth observations, reducing reliance on dense images, appearance cues, watertight meshes, or manual correspondences. It can support future systems that need robust 3D understanding in dynamic real-world environments, such as robotics, augmented reality, and embodied AI.