The Case for Separated Roles in Agentic Research

Modern AI agents are increasingly tasked with the full lifecycle of quantitative factor research: proposing investment factors, backtesting them, selecting survivors, and retiring underperformers. This paper argues that this end-to-end autonomy is fundamentally flawed because it allows the agent to influence the criteria by which its own work is judged. The authors propose a "governed self-evolution" framework where the agent’s role is strictly limited to proposing factors and creating diagnostic probes, while judgment is delegated to a frozen, immutable statistical referee.

The Referee as a False-Discovery Guard

The core of the proposed system is an "anytime-valid" referee. Unlike standard backtesting, which is prone to overfitting and data leakage, this referee scores candidates only on market outcomes revealed after submission. By using a betting-based statistical framework, the referee ensures that false-discovery guarantees hold at any stopping time, regardless of the proposer's policy.

In empirical tests—including synthetic environments with planted truth and a ten-year walk-forward on the CSI 500—the frozen referee admitted 5-11 times fewer sub-threshold factors than "leaky" referees that allowed the agent to influence the judgment process. No proposer (scripted, bandit, or LLM) was able to close this performance gap, proving that the integrity of the referee is the primary determinant of research quality.

Trade-offs and Agentic Capabilities

While the frozen referee significantly improves the reliability of discovered factors, it comes with a cost in speed and short-term performance. An admitted true factor typically requires about 500 trading days of validation, meaning certified portfolios trail behind ungated, high-risk alternatives in terms of Sharpe ratio.

However, the LLM proposer demonstrates unique advantages over traditional scripted or bandit-based approaches. While it matches the bandit in discovery yield, it possesses the distinct capability to author its own diagnostic probes. The authors conclude that the optimal architecture for AI-driven research is to keep the agent focused on creative tasks—proposing and instrument-making—while strictly isolating the judging process to an independent, non-touchable procedure.