Embodied-DETR: End-to-End Temporal 3D Object Detection in Egocentric Views

Embodied 3D object detection is a fundamental perceptual capability for embodied agents, in which observations are partial, heavily occluded, and sequential, requiring modeling of temporal continuity. However, existing benchmarks and methods are primarily designed for fully reconstructed global scenes and fail to capture temporal observation context and instance evolution in first-person perception. We introduce Embodied-Det, a new benchmark for embodied 3D object detection that evaluates detection accuracy, temporal stability, and consistency under egocentric sequential views. Building on this benchmark, we propose Embodied-DETR, an end-to-end temporal detection framework that models scene-level context and instance-level consistency through two complementary temporal modules, Scene-aware Feature Aggregation and Instance-aware Query Embedding. Experiments on Embodied-Det show that existing methods suffer substantial performance degradation in egocentric temporal settings, while Embodied-DETR achieves superior accuracy and temporal consistency, demonstrating the effectiveness of temporal modeling for embodied 3D perception. Codes are available at https://github.com/UniPerceptor/UniPerceptor.

When a robot moves through a room, its camera sees only a narrow, partial view where objects are often hidden or appear only piece by piece. Most existing object detection systems assume the robot can see the whole scene at once, like looking at a blueprint. That doesn’t work in the real world. We built a new test environment called Embodied‑Det that mimics how a robot actually sees: a stream of short, first‑person snapshots. Then we designed Embodied‑DETR, a system that remembers what it saw in past snapshots. By connecting glimpses of the same object over time—like seeing a chair from different angles—it builds a stable, accurate picture even when objects are partly hidden. This makes robots much better at understanding their surroundings. It could help home assistants, delivery drones, or any machine that needs to move safely through dynamic indoor spaces, bringing us closer to truly intelligent, helpful robots.