The Spatial Reasoning Gap

Coding agents excel at text-in/text-out tasks because they operate within the medium they were trained on. However, they struggle significantly with 2D spatial tasks, such as aligning UI elements or manipulating objects on a canvas. To bridge this gap, agents must be taught to interpret the canvas not just as a visual screenshot, but as structured data (JSON). By providing both visual context and underlying data, agents can perform complex spatial actions, such as positioning objects or creating animations, with high precision.

From Single-Shot Prompts to Agentic Workflows

Moving beyond simple one-off commands requires an "agent harness" that allows models to act autonomously. By treating the canvas as a workspace, agents can:

• Self-Direct: Set their own to-do lists and navigate the viewport to locate objects.
• Collaborate: Use multi-agent systems where one agent acts as an orchestrator, assigning tasks to others and waiting for reviews.
• Visualize State: Use whimsical representations (like "fairies") to make agent status readable at a glance, eliminating the need to parse dense chat logs.

The Canvas as a Coordination Hub

For complex projects, the canvas serves as a powerful dependency graph. By representing tasks as nodes that coding agents can execute, developers can manage entire build processes visually. This approach enables:

• Multiplayer Collaboration: Multiple humans and agents can interact with the same board, adding tasks and reviewing work in real-time.
• Real-World Integration: By exposing the editor instance to local desktop environments, agents can move beyond the browser to control system windows, script local files, or even build ephemeral UIs that interact with the actual operating system.
• High-Level Oversight: The canvas provides a persistent, spatial view of project progress, making it easier to track dependencies and bottlenecks compared to traditional task-tracking software.