The Case Against Monolithic Prompts

Monolithic prompts—where a single LLM call attempts to fetch data, analyze context, and generate strategy—are prone to hallucinations. Because the model lacks access to real-time APIs or external data, it often fabricates information. Moving to a graph-based architecture allows developers to treat the workflow as a series of connected nodes, where deterministic functions handle data retrieval and LLMs handle reasoning.

Core Graph Design Patterns

Graph Engineering relies on specific wiring patterns to manage data flow and execution:

  • Fan-Out & Join: Use fan-out to execute independent tasks (like fetching weather, course data, and fitness stats) in parallel. The Join Node acts as a synthesizer, waiting for all branches to complete before aggregating the results into a single data structure. This eliminates the need for custom merger code or additional aggregator agents.
  • Router Patterns: Choosing between an LLM-based router and a deterministic router is critical for cost and reliability. Use a deterministic router (if/else logic) when the decision criteria are closed and the signal is clear. Reserve LLM routers for open-ended requests where the input is unstructured and requires semantic classification.

Optimizing for Cost and Predictability

Effective graph design follows the principle: Predictable work goes in functions, reasoning goes in the model. By offloading data fetching to standard Python functions, you reduce LLM token costs significantly. In the marathon example provided, the entire workflow—including three parallel data fetches and a final strategy generation—requires only a single LLM call, as the other nodes execute via deterministic code.

When to Use Graph Workflows

Graphs are not always necessary, but they are essential when you need structure and reliability.

  • Static Workflows: If you can map the process before the input arrives, use a static graph.
  • Dynamic Workflows: If the structure depends on the input (e.g., deep research tasks where the number of steps is unknown), use dynamic graph generation to define the shape at runtime.

ADK 2.0 provides the framework to implement these patterns, allowing developers to build production-ready systems that are testable, reliable, and cost-efficient.