ReflFlow: Learning Geometry-Guided Ray Tracing for Dynamic Specular Reconstruction

We present ReflFlow, a novel framework for high-fidelity rendering of dynamic specular scenes by addressing two key challenges: precise reflection direction estimation and physically accurate modeling. To achieve this, we propose a Residual Material-Augmented 2D Gaussian Splatting representation that models dynamic geometry and material properties, allowing accurate reflection ray computation. Furthermore, we introduce a Dynamic Environment Gaussian and a hybrid rendering pipeline that decomposes rendering into diffuse and specular components, enabling physically informed specular synthesis via rasterization and ray tracing. Finally, we devise a coarse-to-fine training strategy to improve optimization stability and promote physically meaningful decomposition. Extensive experiments on dynamic scene benchmarks demonstrate that ReflFlow outperforms prior methods quantitatively and qualitatively, producing sharper and more realistic specular reflections in complex dynamic environments.

Imagine filming a moving toy car with a glossy paint job, then asking a computer to rebuild the scene in 3D so you can walk around it virtually. Today's systems can do this for matte objects, but they fall apart on shiny surfaces — the reflections come out blurry, smeared, or just wrong, especially while things are moving. We built ReflFlow to fix that. The first idea is to represent the scene with thousands of tiny, tilted surface tiles that hug the geometry of each object as it moves, so we always know which direction the surface is facing. The second idea is to actually trace the path of light through the scene — the same way physics works in the real world — instead of guessing what reflections should look like. We train the model gradually, letting it master the matte look first before adding the trickier reflective behavior. The result is much sharper and more believable reflections in 3D reconstructions of moving scenes. This matters for virtual and augmented reality, movie special effects, and robots that have to navigate environments full of glass, mirrors, or wet surfaces.