The Failure of 'Model-First' Vertical AI

Many vertical AI projects fail because engineers treat them like general-purpose software. While engineers can easily judge the quality of generated code, they lack the specialized intuition required to evaluate outputs in fields like finance or pharma. Attempting to use an LLM as a judge is a common mistake; models are statistical predictors that lack the causal understanding required to identify 'alpha' in finance or viable drug candidates in pharma. Because these industries intentionally withhold data to maintain competitive advantages, general-purpose models are fundamentally incapable of reasoning through these niche problems.

The Seven-Step Path to Production

To build vertical AI that actually delivers ROI, builders should follow a disciplined process:

  1. Formulate a Narrow Problem: Avoid general-purpose agents. Define a specific, constrained task (e.g., ranking stocks by capital expenditure rather than 'finding market opportunities').
  2. Curate Proprietary Data: Public data is a commodity. The real value lies in unstructured, internal data—such as failed experiments in pharma or historical trade theses in finance.
  3. Model the Workflow: Encode the mental model of an expert into the prompt, breaking the task into logical, sequential steps.
  4. Implement Observability: Use traces to debug performance. This is the 'easy' part that fits on one screen.
  5. Hire the User: This is the critical pivot. You cannot iterate on what you cannot judge. Hiring a domain expert (e.g., a senior scientist or trader) is the only way to bridge the gap between a demo and a production-ready tool.
  6. Build a Learning Loop: The expert curates sources, refines prompts, and performs error analysis. This loop generates a proprietary dataset of what works and what does not.
  7. Ship for ROI: Only move to production when the tool demonstrably saves time or generates revenue.

AI in the Loop (AITL)

Rather than 'Human-in-the-loop,' vertical AI should be viewed as 'AI-in-the-loop.' The expert remains the final decision-maker, while the AI acts as a high-speed assistant that filters candidates or synthesizes data. The moat is never the infrastructure or the model—both are commodities. The moat is the combination of domain-specific data and the expert judgment required to refine the system's performance over time.