The Shift from RAG to Agentic Pipelines

While RAG systems focus on information synthesis and simple Q&A, agentic workflows involve autonomous decision-making and end-to-end document generation. In RAG, a parsing error might lead to one bad answer; in agentic pipelines, errors compound across every step. Success depends on treating document processing not as a single-shot task, but as a multi-stage orchestration problem where input quality dictates the performance of the entire system.

Optimizing Inputs for Frontier Models

Frontier models struggle with raw PDFs because the format was designed for printing, not reasoning. To improve performance and reduce reasoning token costs, teams should:

  • Use Hybrid Parsing: Combine traditional computer vision (CPU-efficient for layout detection) with Vision Language Models (VLMs) for semantic understanding.
  • Implement Agentic OCR: Instead of regenerating text, use token-level corrections to fix common OCR errors (e.g., swapping '0' for 'O').
  • Structure Data for Retrieval: Embedding models struggle with raw HTML or complex table blobs. Create dual representations: an HTML/Markdown version for the LLM to reason over, and a natural language summary specifically designed for the embedding model to improve retrieval accuracy.
  • Classify and Split: Use classification and splitting as an orchestration layer to prevent large, irrelevant documents (like 100-page mail packets) from distracting the model.

Agent Harnesses and Evaluation

Complex tasks like converting line charts to data tables cannot be solved in a single shot. Agent harnesses—where a parent agent manages sub-agents with specific tools like code interpreters—are necessary to bridge the gap between precision and recall. These harnesses allow for iterative self-correction and validation against specific criteria.

Finally, evaluation must be granular. The best teams do not rely on macro-level benchmarks; they implement evals at every stage of the pipeline, from initial parsing to final output, and treat their document corpus as a dynamic file system rather than a fixed, static pipeline.