The Performance Engineering Bottleneck

Performance engineering at scale is often a manual, tedious, and reactive process. Engineers typically profile production services, download raw JSON call stacks, and spend hours in visualizers hunting for bottlenecks. Because this process is so labor-intensive, it is rarely performed until a crisis occurs at 2 AM. As AI coding assistants accelerate code generation, they often produce code optimized for speed of authorship rather than runtime efficiency, leading to an accumulation of technical debt and increased compute costs.

The Agentic Profiling Loop

Rajat Shah and his team at Netflix developed an AI-driven loop to automate this process. The system treats profiling data as a universal language, regardless of the underlying runtime (Java, Python, Go). By feeding call stack data into an LLM, the agent can identify common performance anti-patterns—such as O(N²) loops, redundant object allocations, or loop invariants—without needing to manually inspect the codebase.

Once a bottleneck is identified, the agent:

  1. Clones the specific git repository at the exact commit running in production.
  2. Traces the call path to the source line.
  3. Proposes a code fix.
  4. Validates the fix against a canary deployment (a real-world traffic split).

Building a Shared Pattern Catalog

A critical insight from this work is that performance issues are rarely unique. When the agent identified an anti-pattern in one service, it was able to search across other repositories and find the same issue in seven different services. To scale this, the team built a centralized, ever-growing "Pattern Catalog" stored as simple markdown files in a git repository. This catalog serves as a shared memory for AI agents, allowing them to learn from past findings and apply those lessons to future profiling sessions. This shifts the workflow from reactive (fixing production issues) to proactive (catching anti-patterns during code review).

Trust, Verification, and Human-in-the-Loop

Despite the agent's autonomy, the team maintains a strict human-in-the-loop requirement for production changes. The agent does not push code; it submits a code review. To minimize noise and ensure safety, the agent performs its own pre-flight checks:

  • Functional Correctness: Running existing unit and integration tests to ensure the optimization doesn't break business logic.
  • Canary Validation: Comparing CPU usage, latency, and error rates between the old and new code on live traffic.
  • Observability Reports: Generating a summary of performance gains (e.g., CPU reduction) and potential regressions for the human engineer to review.

Key Takeaways

  • Treat Profiling as Data: Profiling output is a structured, universal language that LLMs can parse to identify performance smells.
  • Build a Centralized Catalog: Don't let performance findings stay siloed. Create a shared repository of patterns and anti-patterns that agents can reference to avoid re-discovering the same issues.
  • Canary is Ground Truth: Never trust an LLM's assessment of performance. Use automated canary deployments to verify actual CPU and latency improvements in production.
  • Shift Left: Use the pattern catalog to inform coding agents during the development phase, catching anti-patterns before they ever reach production.
  • Maintain Human Oversight: When modifying production code, the agent should act as an assistant that prepares the fix and the evidence, but the final decision remains with a human engineer.