From Manual Outreach to Deterministic Workflows

Niels Rogge’s role at Hugging Face involves identifying research papers with weights hosted on third-party sites (Google Drive, Zenodo) and encouraging authors to publish them on the Hugging Face Hub. Initially, he performed this outreach manually. To scale, he first built a deterministic workflow using Python scripts and LLM APIs, triggered by nightly GitHub Actions cron jobs. This approach prioritized predictability and control, avoiding complex agent frameworks in favor of a simple, step-by-step pipeline that replicated his manual process: finding GitHub URLs, reading READMEs, checking for existing artifacts, and opening issues or pull requests.

Transitioning to Autonomous Agent Loops

As the volume of papers increased, Rogge evolved his approach to a fully autonomous agent loop. Unlike the deterministic workflow, this system uses the Claude Agents SDK, a sandbox environment, and the Hugging Face CLI as its primary tool. The agent is deployed using Modal, which allows for massive parallelization by spinning up individual containers for each GitHub issue. This architecture allows the agent to handle follow-ups, documentation, and metadata tagging autonomously. Rogge notes that while he does not disclose the use of an agent to researchers, the outreach remains effective because the agent mimics his original, helpful tone, resulting in only two negative interactions out of thousands of issues.

Evaluation and Best Practices

Rogge emphasizes that the choice between a deterministic workflow and an autonomous agent is not binary; they exist on a spectrum. His current setup leverages open models like Qwen 2.5, which he finds competitive with closed-source alternatives. He stresses the importance of observability and evaluation, recommending tools like Langfuse for tracing prompts, cost, and latency. To avoid generating "slop," he points to Hamel Husain’s work on LLM evaluation as the gold standard for ensuring agent quality. His success demonstrates that modern agents can replace thousands of lines of custom code with simple, skill-based architectures.