Why BM25 Remains the Optimal Retrieval Primitive for AI Agents

The Shift from Human to Agentic Search

BM25 (Best Match 25) has regained relevance not because the algorithm changed, but because the user profile has fundamentally evolved. Human search queries historically consist of two or three words, whereas LLMs generate long, highly specific queries that utilize complex syntax, operators, and entity-rich terminology. Because LLMs possess vast internal knowledge of companies, dates, and product identifiers, they are significantly more effective at exploiting the exact-match capabilities of lexical scoring than human users. Reasoning is rarely the bottleneck in agentic tasks; rather, the constraint is the context window, which acts as a modern-day floppy disk. Effective retrieval is required to curate the information that enters this limited space.

Optimizing Retrieval for Agentic Workflows

Rather than relying on traditional information retrieval metrics like NDCG, which focus on single-shot query performance, developers should evaluate retrieval based on end-to-end task success. A promising pattern involves treating retrieved documents as a file system workspace. By exposing documents to the agent as a structured file set, the agent can leverage its existing training in coding primitives like grep, sed, and ripgrep to navigate and filter information. This approach provides several advantages:

• Explainability: Because BM25 relies on literal term matching, the agent can interpret why a specific document was retrieved, allowing it to refine its query strategy iteratively.
• Efficiency: BM25 is computationally inexpensive compared to high-parameter embedding models, which require significant infrastructure for inference.
• Parameter Sensitivity: The effectiveness of BM25 is highly dependent on hyperparameter configuration. Standard baselines in research often perform poorly because they use default settings that are not optimized for the specific document corpus or the long-form queries generated by agents.
• Tool Integration: Combining BM25 with sandbox-based file system access allows agents to perform progressive disclosure, where they initially inspect snippets and only pull full documents into the context window when necessary.