Optimizing Communication via Belief Divergence

In multi-agent reinforcement learning (MARL), agents often struggle with the trade-off between the high bandwidth costs of constant communication and the performance degradation caused by isolated decision-making. This research proposes a principled gating mechanism that allows agents to decide when to communicate based on the information gain they expect to provide to their peers.

Instead of relying on heuristic-based gating or constant broadcast, the authors model communication as a decision triggered by the divergence between an agent's internal belief distribution and the predicted belief distribution of its peers. By calculating the Kullback-Leibler (KL) divergence between these distributions, the system identifies moments of high uncertainty or information asymmetry. If the divergence exceeds a learned threshold, the agent broadcasts its state or intent; otherwise, it remains silent, conserving bandwidth.

Balancing Coordination and Efficiency

This approach effectively treats communication as a scarce resource. The core insight is that communication is only valuable when it significantly updates the belief state of other agents in a way that improves collective task performance.

Key advantages of this framework include:

  • Reduced Bandwidth: By suppressing redundant or low-value messages, the system significantly lowers the communication overhead in multi-agent environments.
  • Adaptive Coordination: The gating threshold can be tuned to prioritize either strict coordination or communication efficiency, depending on the constraints of the environment.
  • Robustness: Because the mechanism is grounded in belief distributions, it is inherently more stable than methods that rely on simple thresholding of raw observation changes, as it accounts for the actual state-space uncertainty of the agents involved.

The authors demonstrate that this method maintains high performance in complex coordination tasks while drastically reducing the total number of messages exchanged compared to standard broadcast architectures.