The Role of Second-Order Theory-of-Mind in Collaboration

Effective human-autonomy collaboration often fails not because of a lack of technical capability, but because of a mismatch in mental models. The authors argue that for an autonomous agent to be a truly effective teammate, it must possess 'second-order Theory-of-Mind' (ToM). While first-order ToM involves an agent modeling what a human believes about the world, second-order ToM requires the agent to model what the human believes about the agent itself.

By maintaining this recursive belief structure, the agent can anticipate when a human teammate has an incorrect perception of the agent’s current state, intentions, or capabilities. This allows the agent to proactively correct these misconceptions before they lead to task failure or safety incidents.

Synchronizing Beliefs to Improve Team Performance

The core contribution of the framework is a mechanism for belief synchronization. The agent continuously evaluates the discrepancy between the human’s inferred belief about the agent and the agent’s actual internal state. When this discrepancy exceeds a specific threshold, the agent triggers a communication or action designed to realign the human's mental model.

This approach shifts the burden of communication from the human to the agent. Instead of the human having to constantly query the agent's status, the agent acts as an active participant in maintaining shared situational awareness. The authors demonstrate that this recursive modeling significantly reduces the cognitive load on human teammates and improves the overall efficiency of the human-autonomy team in complex, dynamic environments where information asymmetry is common.