Disentangling Shared and Private Information

Multimodal sentiment analysis often suffers from data incompleteness, where one or more modalities (e.g., audio, video, or text) are missing during inference. The MIDAS (Mutual Information Disentanglement with Uncertainty-Aware Fusion) framework addresses this by separating multimodal representations into two distinct components: shared information, which is common across modalities, and private information, which is unique to a specific modality. By disentangling these features, the model ensures that the shared representation remains robust even if specific modalities are absent, as the shared core captures the underlying sentiment signal that persists across different data streams.

Uncertainty-Aware Fusion for Robust Prediction

Beyond disentanglement, MIDAS employs an uncertainty-aware fusion mechanism to handle the noise and variability inherent in incomplete data. When modalities are missing or degraded, the model estimates the uncertainty associated with each available feature. Instead of treating all inputs with equal weight, the fusion process dynamically adjusts based on the confidence level of the available modalities. This prevents the model from relying on unreliable or incomplete data streams, effectively mitigating the performance drop typically seen in multimodal systems when input data is sparse. By integrating these uncertainty estimates, MIDAS maintains high predictive accuracy across varying degrees of modality absence, proving more resilient than traditional fusion techniques that assume complete input availability.