The Failure of Prediction-Accuracy as a Metric

Large language models (LLMs) are increasingly used to synthesize Code World Models (CWMs)—executable representations of environment rules that classical planners use to navigate tasks. Current evaluation standards typically rely on "sampling gates," where a model is accepted if it achieves high transition accuracy on sampled trajectories. This research demonstrates that this metric is fundamentally flawed for planning.

Even models achieving 100% transition accuracy and >98% state-accuracy on a planner's search distribution can fail systematically during actual play. The issue is that the <1% of rules the model gets wrong are often the most pivotal dynamics. The study quantifies this "verified-vs-correct gap," noting that the play cost of these omitted rules is significant (0.091, with a 95% CI of 0.065, 0.117).

The Quantitative Law of Model Danger

The paper establishes that the harm caused by these models follows a specific quantitative law: danger = play_cost * (1 - rarity)^N. In this formula, the (1 - rarity)^N term acts as a "gate-miss factor." This proves that even if a model is highly accurate on common states, it will inevitably fail if it misses rare but critical rules, as these rules are essential for successful planning. The failure is not a result of insufficient data; the author argues that LLM synthesis functions as "rule translation" rather than true "rule inference." Across various models (including GPT-5.x) and training regimes (such as DAgger), models failed to infer the omitted rules.

Implications for Planning-Oriented Models

The same mechanism of failure appears in the belief-inference functions of imperfect-information CWMs. The author proves a coverage bound where a gate is only identifying when N >= b^d_max. This explains why simple games like Kuhn poker do not exhibit this gap, whereas more complex environments do. The author demonstrates this by constructing "Beacon," an inference function that passes verification gates but loses every game it plays.

Ultimately, the research suggests that adequacy for world models should not be measured by prediction accuracy on sampled transitions. Instead, developers must measure performance directly on the search distribution or through actual play to ensure the model captures the rules that actually dictate success.