The Trap of Token Economics

Most AI-powered products fall into a 'token trap' where they rely on frontier models for every task, leading to unsustainable costs. As model providers frequently upgrade models, deprecate predecessors, or change pricing, businesses that lock themselves into a single vendor lose their leverage. Sarah Sachs argues that treating model providers as competitors is essential; if you are reselling tokens from a lab that also sells a first-party product, you are paying a markup on a markup, which is a business model that is difficult to defend.

The Model-Agnostic Playbook

To build a sustainable AI product, companies must shift from 'winning on token economics' to 'winning on product.' This requires:

  • Model Agnosticism: Build systems that can route traffic to the most cost-effective model for the specific task. Notion uses an 'auto-model' that handles 75% of traffic, dynamically switching providers to optimize for cost, capability, and latency.
  • Capability-Based Routing: Not all tasks require frontier models. Triage tasks or simple data analysis should be routed to smaller, cheaper models. Use frontier models only for complex reasoning tasks that truly demand them.
  • Leveraging Open Weights: Open-weight models are increasingly capable and provide critical negotiation leverage against the oligopoly of frontier labs. They serve as a credible alternative that puts downward pressure on pricing.
  • Deterministic Alternatives: Avoid using LLMs for tasks that can be solved with standard code. If a task involves deterministic SQL queries, CSV-to-PDF conversion, or CLI tool calls, use CPUs instead of GPUs. This prevents 'token poverty' and improves system reliability.

Orchestration and Security

The future of AI-powered products lies in multi-agent orchestration rather than simple prompt-response loops. However, this introduces the 'lethal trifecta' of risk: access to private data, exposure to untrusted content, and the ability to communicate externally. To mitigate this, builders must focus on:

  • Sandboxing and Determinism: Ensure agents operate within controlled environments where their actions are inspectable and reversible.
  • Persistence: Treat AI interactions as part of a durable system of record (like a collaborative document) rather than ephemeral markdown files. This allows human-in-the-loop collaboration where agents can tag teammates, scope tasks, and open PRs.
  • Eval-Driven Development: Invest in granular evaluation of entire task trajectories rather than single-call latency. This expertise allows companies to make informed decisions and provides valuable feedback to model labs, which can be used as a bargaining chip in contract negotiations instead of relying solely on volume-based discounts.