The Shift from Token Maxing to Value Maxing

Traditional software eras developed specific control surfaces: usage caps for SaaS and autoscaling for cloud. The agentic era currently lacks a native control layer at the point where code calls an LLM. Current solutions like model gateways operate at the request level, which is insufficient for managing complex agentic workflows that involve recursive loops, sub-agent spawning, and context bloat. The goal is to move from "token maxing" (unbounded exploration) to "value maxing" by attributing costs to specific agent runs and enforcing policies in-place.

Architecture: Boundary, Governor, and Control Plane

The proposed system, Token Ops, uses an out-of-band control plane to manage costs without requiring significant code rewrites. It consists of three primary components:

  • Instrumentation (The Bridge): Uses a @boundary annotation on existing methods to track inputs/outputs and float telemetry to the control plane. This acts as a two-way channel, allowing the control plane to push instructions back to the agent.
  • The Governor: A local module that receives authorized actions from the control plane and applies them to the agent. It ensures the control plane only executes pre-approved, non-destructive interventions.
  • Control Plane: Groups runs into segments based on custom dimensions (e.g., user cohorts), maintains a ledger of traces, and evaluates policies against budgets.

Steering vs. Halting

Instead of relying solely on circuit breakers (halting), the system introduces "steering" to keep agents within budget:

  • Halt: A last-resort action that kills the agent run if it exceeds a hard budget.
  • Steer: A proactive approach where a "cost guard" monitors both total budget consumption and velocity. If an overrun is predicted, the system injects instructions into the agent's system prompt (e.g., "be more succinct" or "limit tool output chunks").

In benchmarks on open-source repositories like browser-use and MetaGPT, this approach reduced average spend by approximately 78% while increasing the successful completion rate of agent runs from 67% to 96% compared to simple throttling.