Most commerce agents fail because they act as simple wrappers around a search bar, assuming users have well-defined intent and the correct vocabulary. In reality, users arrive with a "vibe"—a fuzzy, subjective goal. The agent's primary job is to bridge this articulation gap through proactive elicitation and hand-holding, rather than forcing the user to perform the heavy lifting of keyword refinement.

The Three-Phase Commerce Loop

To handle fuzzy intent, agents should operate in a continuous loop of discovery, research, and response:

  • Discovery: The agent builds a working state by aggregating past conversations, user context, and reference images. It distinguishes between hard constraints (e.g., budget) and soft constraints (e.g., style). Crucially, it identifies which variables must be refreshed in real-time (like inventory) to ensure recommendations remain valid.
  • Research: Instead of a static checklist, the agent uses "expected information gain" to prioritize the next question. It asks the single most impactful question first (e.g., "How wide is the room?") to prune the search space. It also employs multimodal elicitation—using visual boards rather than text when preferences are subjective—to establish a common language with the user.
  • Response: The agent must adapt its output format to the user's current need. A policy question might require a summary, a product comparison requires a table, and style exploration requires visual inspiration. The format is a core part of the agent's intelligence, not just a UI preference.

Evaluating Agent Performance with Auto-Raters

Reliability is maintained through auto-raters at every stage of the loop. Key evaluation techniques include:

  • Counterfactual Sensitivity: The system flips parts of a user query to ensure the agent's extracted constraints change or remain stable appropriately.
  • Format Accuracy: Auto-raters verify that the chosen response format (table vs. list vs. image) is optimal for the user's specific query.
  • Turn Efficiency: The system penalizes over-asking, measuring how quickly the agent can uncover hidden preferences without exhausting the user.
  • Data Fidelity: Ensuring the model does not hallucinate and that all retrieved product information maps accurately to the merchant's ontology.