Automating Evidence-Based Testing

Cognition is leveraging GPT-6 Astra to enhance Devin’s ability to verify its own output. Rather than relying solely on code generation, Devin now uses Astra to execute software in simulated environments. For example, when tasked with testing an application like the iPhone game Otter Run, Devin provides engineers with a video recording of the software running in a simulator alongside a structured report. This report explicitly details which test checks passed and identifies specific areas that remain untested, allowing human engineers to focus their attention on the gaps rather than verifying the entire codebase.

Reducing Manual Code Review

The primary goal of this integration is to shift the engineering workflow away from manual code review toward a model of evidence-based verification. By having the AI provide visual and documented proof that a change functions as expected, Cognition aims to decrease the time engineers spend inspecting routine code changes. This capability also extends to bug resolution: when a customer reports an issue via screenshot, Devin uses Astra to diagnose the bug, apply a fix, and return a new screenshot demonstrating the successful result. This loop enables faster turnaround times and higher shipping velocity by minimizing the cognitive load required for routine verification.