The Shift from Knowledge to Agency

Modern AI development has moved beyond testing models on static knowledge benchmarks toward evaluating agentic autonomy. The primary barrier to long-horizon agent reliability is not model architecture, but the quality of the data and reinforcement learning (RL) environments used during post-training. Bespoke Labs argues that RL environments are essentially a specialized form of data, and treating them with the same rigor as supervised fine-tuning (SFT) sets is critical for production success.

Curation Recipes and Counterintuitive Lessons

Through the development of the OpenThoughts reasoning dataset, the team identified that a disciplined curation stack is the primary driver of model capability. Key findings include:

  • Diversity over volume: Sampling multiple reasoning traces for a single question is more effective than simply increasing the number of unique questions. This diversity in reasoning paths improves model robustness.
  • Teacher selection: Stronger models are not always the best teachers. Specific models (e.g., Qwen) often outperform larger, more general models (e.g., Claude) in specific reasoning tasks.
  • SFT remains foundational: While RL is powerful for final performance gains, SFT provides the bulk of the capability improvements for most enterprise use cases.
  • Task-specific tagging: In production scenarios, such as credit card compliance, adding structural tags to data helps models focus on compliance rules rather than hallucinating specific numerical values, significantly improving metrics and reducing latency.

The Emerging Post-Training Stack

Building reliable agents requires a multi-layered infrastructure stack that goes beyond simple model training. This includes:

  • Environment Orchestration: Tools to manage sandboxes, generate rollouts, and handle checkpointing/snapshotting for long-horizon tasks.
  • Curation Tooling: Systems like Curator that manage the pipeline from source prompts to filtered, high-quality training data.
  • Prompt Optimization: Integrating techniques like Japa (LLM-based prompt optimization via reflection) to refine system prompts and harnesses alongside model weights.
  • Measurement: Tracking versioning and quality metrics across both data and environment configurations to ensure reproducible improvements.