Shift from Code-Fixing to System-Building

The core challenge in adopting AI agents is moving beyond individual developers "vibing" with prompts toward systemic engineering. Instead of manually fixing agent-generated code, teams should focus on improving the system that produces it. This involves building harnesses, loops, and context-rich environments that allow agents to operate autonomously. When an agent fails, the retrospective should not focus on the specific code error, but on the systemic wall the agent hit, leading to a fix that prevents the issue for the entire team.

The Two-Metric Framework for Productivity

Measuring AI success by token spend or individual output is misleading. Instead, track two specific metrics to gauge organizational maturity:

  1. Human Touches: The number of manual interventions required to get the agent to produce the correct result. This should trend downward as harnesses and context improve.
  2. Reuse Rate: The degree to which a fix or improvement is shared. A single improvement to a common harness or context registry provides a multiplier effect, benefiting every developer rather than just making one person "10x."

Platform Engineering and Paved Roads

As AI adoption spreads, organizations face a "sprawl" problem where teams reinvent authentication, security, and linting. The platform team must transition from infrastructure-only roles to managing a registry of reusable skills, guardrails, and context.

  • Paved Roads: Rather than letting "a thousand flowers bloom," provide 3-4 standardized, maintained paths for AI development.
  • Visibility: Make AI spend visible to teams. When developers see the cost of their agent iterations, they are naturally incentivized to optimize their prompts and harnesses to reduce unnecessary cycles.

Organizational Strategy and Hiring

Scaling AI is an organizational transformation, not a technical one. Avoid generic "champions programs" or hackathons. Instead, empower team leads to mandate reusable context and standardized engineering practices. When hiring, ignore generic titles like "AI Engineer." Instead, test for:

  • Taste: The ability to explain why a specific AI-generated solution is good.
  • Collaboration: A willingness to share knowledge and contribute to the common harness rather than working as a solo player.
  • Engineering Fundamentals: The ability to apply traditional software practices (testing, documentation, security) to agentic workflows.