The Rise of the 'Super-User'

BM25 (Best Match 25) is a 30-year-old lexical scoring function that has regained relevance not because the algorithm changed, but because the user did. Historically, human search queries were short (2-3 words). Modern LLMs, however, possess vast internal knowledge of entities, dates, and identifiers. They act as 'super-users' capable of generating long, highly specific queries and executing them in rapid, iterative loops. Because LLMs are trained to excel at coding and shell primitives, they are uniquely adept at using lexical search tools to navigate information.

Reasoning vs. Retrieval Bottlenecks

Research using the BrowseComp+ benchmark (830 riddle-like questions over 100,000 documents) demonstrates that reasoning is rarely the bottleneck for modern models. When relevant evidence is manually placed in the context window, accuracy remains high. Accuracy only drops when the model is forced to use a retrieval tool, proving that the quality of the search harness and query formulation is the primary constraint. Since context windows—which the author compares to 1.4MB floppy disks—remain limited, the ability to selectively retrieve and filter information is essential.

The Case for Lexical Primitives

BM25 remains a powerful primitive for three reasons:

  1. Explainability: Because it relies on literal term matching, the model can easily inspect why a result was returned and adjust its next query accordingly.
  2. Precision: It excels at matching specific entities, SKUs, and codes that dense embedding models often struggle to represent accurately.
  3. Efficiency: It is computationally inexpensive and requires less infrastructure than high-parameter embedding models.

Scaling via Dynamic Workspaces

The author advocates for a 'file system' approach to retrieval. Instead of forcing the model to process a single ranked list, developers should dump retrieved documents into a workspace. This allows the model to use familiar tools like grep, sed, and awk to perform progressive disclosure—reading titles and snippets first, then deciding which documents to fully ingest. This paradigm aligns with how frontier models are already being trained to interact with code and bash environments, ensuring that search infrastructure improves in tandem with model capabilities.