The Infrastructure Gap in Agentic Systems

As agentic systems evolve from simple request-response loops to persistent, asynchronous entities, the infrastructure requirements shift significantly. Current agent SDKs excel at prototyping but fail to provide the distributed primitives necessary for production. Building these primitives—retry logic, recovery, session isolation, and cancellation—from scratch is error-prone and complex. The core challenge is moving from "stateless" execution to "durable" execution, where a process can survive crashes, redeploys, and long-duration waits (e.g., waiting weeks for human approval) without losing state or restarting from scratch.

Durable Execution via Event Journaling

Restate acts as a server-side proxy that sits in front of agent services, maintaining a lifeline connection. It uses an event-driven distributed log, inspired by Apache Flink and Meta’s event infrastructure, to journal every step of an agent's execution.

  • Resilience: If a process fails, the system replays the journal to return the agent to its exact state of failure rather than starting over.
  • Push-based Architecture: Unlike traditional workflow orchestrators that rely on polling, Restate pushes invocations to services. This results in significantly lower latency (e.g., 45ms p99 for a 10-step workflow) and better compatibility with serverless environments.
  • Suspension: By treating functions as durable, the system can "suspend" execution during long waits (like human approval), consuming zero serverless execution time until the process is unblocked.

Virtual Objects and Stateful Control

Beyond simple workflows, Restate models agents as "Virtual Objects"—stateful actors with unique session IDs and isolated key-value stores. This architecture enables complex interaction patterns:

  • Signaling: Because sessions are persistent, developers can inject new information into a running agent loop mid-flight. A classifier can determine if the new input is relevant, triggering a signal to the loop or a complete cancellation and restart.
  • Cancellation Propagation: When a controller is cancelled, the signal propagates down the call chain, effectively "rewinding" the stack and killing sub-agents to prevent wasted resources.
  • Flow Control: By abstracting LLM calls into dedicated handlers, teams can implement policy checks and rate limiting across the entire agentic platform, preventing cost overruns and ensuring consistent resource allocation.