DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories

Existing multimodal retrieval systems excel at semantic matching but implicitly assume that query-image relevance can be measured in isolation. This paradigm overlooks the rich dependencies inherent in realistic visual streams, where information is distributed across temporal sequences rather than confined to single snapshots. To bridge this gap, we introduce DeepImageSearch, a novel agentic paradigm that reformulates image retrieval as an autonomous exploration task. Models must plan and perform multi-step reasoning over raw visual histories to locate targets based on implicit contextual cues. We construct DISBench, a challenging benchmark built on interconnected visual data. To address the scalability challenge of creating context-dependent queries, we propose a human-model collaborative pipeline that employs vision-language models to mine latent spatiotemporal associations, effectively offloading intensive context discovery before human verification. Furthermore, we build a robust baseline using a modular agent framework equipped with fine-grained tools and a dual-memory system for long-horizon navigation. Extensive experiments demonstrate that DISBench poses significant challenges to state-of-the-art models, highlighting the necessity of incorporating agentic reasoning into next-generation retrieval systems.

When people search their own photo albums, they usually think in terms of experiences, like "the trip where it rained the whole time" or "the day after we watched the fireworks." But current photo search only matches faces, places, dates, and objects against each photo individually. It cannot follow a narrative thread across a collection to figure out which photos you mean. We propose DeepImageSearch, a new paradigm where AI acts as an agent that actively explores a photo collection step by step, connecting clues scattered across different events to answer these experience-based queries. We built DISBench, a benchmark of 122 such queries over real personal photo albums, and ImageSeeker, a baseline AI assistant equipped with search, filtering, and visual inspection tools. We found that DISBench poses significant challenges to today's strongest AI models, especially when the answer requires linking evidence across separate events rather than searching within a single one. DISBench provides the first concrete testbed for measuring and advancing this capability.