The Shift to the 'Nearly Headless' Web

For two decades, the web has been defined by browser tabs and proprietary dashboards. This model is becoming obsolete as personal AI assistants replace browsers as the primary entry point for users. The future is a 'nearly headless' web: agents interact with services via APIs, but pull back specific UI 'atoms'—small, functional components—to present to the user. This allows users to complete tasks (like booking travel or managing projects) without ever visiting a traditional website or learning a unique dashboard interface.

Why Static Specifications Fail

There is a disconnect between how developers prepare for agents and how agents actually behave. While many websites publish llms.txt files to guide AI, research shows that most agents ignore them, preferring to navigate directly to documentation or homepages. The author argues that developers should stop guessing what agents need. Instead, they should use tools to observe thousands of agent interactions to identify the actual paths and data requirements agents prioritize. Best practices for AI integration are evolving too quickly for static documentation; real-world agent feedback is the only reliable source of truth.

The New Rules of Brand and Discovery

In an agentic world, brand loyalty is secondary to technical integration quality. If a coding agent recommends a service because it has a superior MCP (Model Context Protocol) implementation or API, users will switch, regardless of a decade of brand history. Furthermore, products that force an agent to 'spin up' a browser to function will be penalized and eventually bypassed.

To survive, companies must focus on:

  • Agent Readiness: Ensuring APIs and MCP servers are robust and well-documented for machine consumption.
  • Accessibility: Agent accessibility and human accessibility are effectively the same work. Both rely on clear, semantic signals that allow a non-visual entity to navigate and understand the interface.
  • Discovery Layers: As the web grows, we need a new discovery layer for AI resources. Traditional SEO is insufficient. Emerging standards like AI-catalog.json and the Agentic Resource Discovery (ARD) standard are essential for allowing agents to programmatically find and query the right tools for a specific intent.