The Problem of Paper-Code Discrepancy

In the field of machine learning, the gap between published research papers and their corresponding code implementations is a significant barrier to reproducibility. Researchers often encounter inconsistencies where the mathematical formulations or algorithmic descriptions in a paper do not align with the actual implementation provided in a repository. The 'Dude' (Dual-Detection) system is a multi-agent framework specifically engineered to automate the detection of these discrepancies, ensuring that codebases accurately reflect the claims made in academic literature.

The Dual-Detection Multi-Agent Architecture

Dude operates by leveraging a multi-agent system that performs a two-pronged analysis of the research artifacts:

  1. Paper-to-Code Analysis: The system extracts key algorithmic components, hyper-parameters, and architectural details from the PDF of the paper. It then maps these against the repository structure to identify missing or mismatched logic.
  2. Code-to-Paper Analysis: The system analyzes the codebase to infer the underlying logic and compares it against the stated methodology in the paper.

By using a dual-detection approach, the framework reduces false positives and ensures that both the high-level intent of the research and the low-level implementation details are scrutinized. The agents are designed to communicate, cross-referencing their findings to pinpoint specific lines of code or sections of text that deviate from one another. This collaborative agentic workflow allows the system to handle complex, multi-file repositories that would be difficult for a single-pass LLM to analyze effectively.

Implications for Reproducibility

Accepted to EMNLP 2026, the Dude framework represents a shift toward automated verification in scientific publishing. By providing a structured way to audit code against text, it helps maintain the integrity of research findings. For builders and researchers, this tool offers a path to verify the claims of open-source projects before integrating them into production pipelines, effectively acting as a 'sanity check' for the technical claims made in research papers.