FreeRet: MLLMs as Training-Free Retrievers

Multimodal large language models (MLLMs) are emerging as versatile foundations for mixed-modality retrieval. Yet, they often require heavy post-hoc training to convert them into contrastive encoders for retrieval. This work asks: \textit{Can off-the-shelf MLLMs serve as powerful retrievers without additional training?} We present \textbf{FreeRet}, a plug‑and‑play framework that turns any MLLM into a two‑stage retriever. FreeRet first derives semantically grounded embeddings directly from the model for fast candidate search, and then exploits its reasoning ability for precise reranking. The framework contributes three advances: bypassing lexical alignment layers to obtain semantically faithful embeddings, conditioning representation generation with explicit priors, and mitigating framing effect in reranking via neutral choice framing. On the MMEB and MMEB-V2, FreeRet substantially outperforms models trained on millions of pairs. Beyond benchmarks, FreeRet is model-agnostic and scales seamlessly across MLLM families and sizes, preserves their generative abilities, supports arbitrary modality combinations, and unifies retrieval, reranking, and generation into end-to-end RAG within a single model. Our findings demonstrate that pretrained MLLMs, when carefully harnessed, can serve as strong retrieval engines without training, closing a critical gap in their role as generalists.

Problem: MLLMs can understand text, images, and videos, but using these models to search across different types of media usually requires expensive and heavy training on massive datasets. This costly process limits their scalability and makes it hard to quickly adapt new models to everyday search tasks. Solution: We introduce FreeRet, a framework that turns existing models into powerful search engines without any extra training. We achieve this by bypassing the specific internal layers, extracting its deep understanding of the core content instead. Furthermore, we reframe the reranking process into a simple multiple-choice question. Impact: FreeRet allows any standard, off-the-shelf MLLM to instantly become a highly accurate multimodal retriever. It significantly outperforms many models that were heavily trained on millions of data examples. This work paves the way for a future where a single, general-purpose AI can seamlessly handle both complex reasoning and searching tasks.