The Limitation of Traditional Agentic Search
Agentic search typically relies on retriever-mediated interfaces like BM25 or ColBERT. While these are effective for candidate discovery, they treat documents as static, bounded results. This prevents agents from performing complex operations—such as cross-document verification, filtering, or reorganization—that require a more flexible, interactive environment. While Direct Corpus Interaction (DCI) allows for shell-executable commands, it becomes unstable and slow as the corpus size increases, making it impractical for large-scale data.
The DR-DCI Framework
DR-DCI (Retriever-steered DCI) solves this by treating retrieval as an agent-callable action that populates a local, dynamic workspace. Instead of forcing the agent to operate on the entire corpus, the agent retrieves relevant documents and pulls them into a local environment where it can execute DCI operations. This architecture provides two distinct advantages:
- Scalability: By limiting the scope of DCI operations to a retrieved subset, the system remains performant even as the total corpus grows to 10M+ documents.
- Precision: The agent retains the ability to perform complex, terminal-style operations (filtering, comparison, verification) on the documents it has selected.
Performance and Scaling Results
Experiments demonstrate that DR-DCI consistently outperforms raw DCI and standard retrieval baselines. On the Browsecomp-Plus benchmark, DR-DCI achieved 71.2% accuracy, which increased to 73.3% when utilizing workspace-preserving context resets. Key findings include:
- Efficiency: The framework reduces tool usage, wall time, and overall computational cost compared to raw DCI.
- Robustness: While raw DCI fails as corpus size increases, DR-DCI maintains effectiveness across scales ranging from 100K to 10M documents.
- Versatility: In a 20M-scale file-per-document setting (Wiki-18 QA), the model achieved an average score of 63.0 across six benchmarks, outperforming existing search-agent baselines.