The Challenge of Incentive Misalignment
Computer-use agents (CUAs) are increasingly tasked with navigating online marketplaces where the platform's incentives—such as prioritizing specific products—often conflict with the user's objectives. While existing benchmarks focus on cooperative tasks or direct adversarial attacks, they fail to measure how agents handle environments that use subtle steering mechanisms to influence decision-making.
Diagnosing Agent Failure Modes
The researchers introduced CAVEAT, a benchmark covering nine marketplace environments and eight common steering mechanisms. They identified three primary failure points where agents succumb to environmental pressure:
- Priority Distortion: The agent allows the environment to redefine or shift the user's original goals.
- Premature Narrowing: The agent restricts its search space too early, ignoring potentially better alternatives.
- Hasty Commitment: The agent makes a final decision before gathering or resolving sufficient evidence to verify the user-optimal choice.
Improving Robustness with CAVEAT-Harness
To address these vulnerabilities, the authors developed CAVEAT-Harness, a framework designed to mitigate these specific failure modes. By forcing the agent to maintain focus on user intent and delaying commitment until evidence is resolved, the harness improved user-optimal purchasing rates by 55.0%. The study demonstrates that while larger models and increased reasoning capabilities offer some protection, they are insufficient on their own; targeted interventions and post-training are required to ensure agents remain robust in environments that do not share the user's goals.