The Problem: The Engineering Bottleneck

In many organizations, business teams rely on engineers to answer data-driven questions. This creates an inefficient loop: a business user asks a question, an engineer writes a SQL query, the user realizes they need more context, and the process repeats. This is slow, unscalable, and results in one-off Slack messages rather than reusable knowledge.

The Solution: Studio and Deterministic Widgets

WorkOS built "Studio," an internal workspace that allows users to query data sources (Snowflake, Linear, Notion) using natural language. The system uses an agentic architecture (powered by LangGraph and Claude 3 Opus) to parse questions, query databases, and generate answers.

The core innovation is the widget system. Instead of relying on an LLM to generate a new query every time a user views a dashboard, the LLM writes declarative JavaScript code that interacts directly with the data sources. Once generated, these widgets are deterministic and do not require further LLM calls to run, making them fast, reliable, and cost-effective.

Reliability and Implementation Tactics

To make the system robust enough for support teams to use, the team implemented three key engineering patterns:

• Just-in-Time Context Injection: Rather than bloating the context window with the entire database schema upfront, the system injects specific schema context only at the moment a tool is invoked. This keeps the context window clean and focused.
• Layered Guidance: The system uses a "guidance layer" that explicitly instructs the model to distrust its internal training data regarding the company's product (which changes rapidly) and instead prioritize primary sources like internal documentation and live database schemas.
• Pre-flight Query Validation: Before a widget is finalized, the system executes the generated SQL against the database to ensure it returns actual data. This prevents "valid" SQL queries that return zero rows from being hardcoded into production widgets.

Governance and Scaling

While the system currently relies on user-based authentication for data sources, the team is moving toward "org connectors" using their own product, Pipes. This allows a single admin to configure a connection and define granular access rules, removing the need for every employee to maintain individual credentials. The team prioritizes model quality (Opus) over cost, noting that because the final widgets are declarative code, the high cost of the initial LLM generation is a one-time investment that pays off in long-term reliability.