The Shift from Contrived to Process-Based Data
Data is the most critical, yet underfunded, input for AI performance. The industry currently suffers from a reliance on 'Type-2' data—contrived, synthetic examples generated by experts in artificial settings. While this helps models reach basic competence, it fails to drive them toward true expertise. The real opportunity lies in 'Type-1' data: process-based, real-world workflows (e.g., GitHub commits, session replays, or professional decision-making traces) that capture how work actually gets done. Because data is a highly depreciable asset, the only durable supply is found in live business partnerships rather than static, historical codebases.
The Verification Bottleneck
Model training success is governed by the 'Verifiers Law,' which states that the ease of training a model is proportional to the task's verifiability. Verifiability is defined by three axes:
- Asymmetry: Can the task be decomposed into checkable steps?
- Veracity: Is there consensus on what 'correct' means?
- Proliferation: How often does the real world provide fresh, verified examples?
Coding succeeded as the first mature AI application because it scored high on all three axes (unit tests, community consensus, and infinite public examples). Current efforts in law, finance, and biology struggle because they score low on these axes, making them reliant on proprietary enterprise data that is difficult to access and verify.
The 'Benchmark Psychosis' and Vendor Misalignment
There is a systemic issue with how the industry measures progress. Many benchmarks are 'quietly fake'—they are created by the same vendors selling the data used to train models to pass them. This creates a Goodhart’s Law scenario where the metric becomes the target, leading to 'benchmark psychosis.' A single benchmark number is merely a noisy sample from an unmeasured distribution. Builders should instead use 'cross-harness differencing' and rubric-based analysis to understand how models actually perform on long-horizon, non-verifiable tasks, rather than relying on leaderboard scores.
Building Durable Moats in an Unstable Market
History shows that infrastructure pioneers rarely maintain long-term market dominance. As foundation models become more commoditized and efficient, the real value for builders is not the model itself, but the pipeline that connects real-world work to the model. Successful data companies are pivoting to become 'Antikythera' mechanisms—infrastructure layers that translate messy business contexts into automated evaluation and reinforcement learning (RL) pipelines. The goal for enterprises is to own their own intelligence, using an abstraction layer that allows them to swap base models (e.g., switching from proprietary to open-source) without needing to rebuild their entire post-training data stack from scratch.