The Challenge of Subjective XAI Evaluation
Explainable AI (XAI) is critical for trust and transparency, yet evaluating the quality of an explanation remains a notoriously difficult task. Traditionally, this requires expensive, time-consuming human studies to assess factors like faithfulness, clarity, and utility. The XAI-Arena framework explores a scalable alternative: using LLMs as automated evaluators to judge the quality of XAI explanations across various domains.
The XAI-Arena Framework
The core premise of XAI-Arena is to treat explanation quality assessment as a comparative or scoring task for LLMs. By providing models with the original input data, the model's prediction, and the generated explanation, researchers can prompt the LLM to evaluate the explanation based on specific criteria. This approach shifts the burden of evaluation from human participants to automated pipelines, potentially allowing for rapid iteration during the development of XAI systems.
Key Research Questions
The study investigates several critical dimensions of LLM-based evaluation:
- Alignment with Human Judgement: To what extent do LLM scores correlate with human assessments of explanation quality? If LLMs can replicate human intuition, they serve as a viable proxy for user-centric evaluation.
- Consistency and Bias: Do LLMs exhibit systematic biases when evaluating explanations, such as favoring longer explanations or those generated by specific model architectures?
- Scalability: How does this automated approach perform across different types of XAI methods (e.g., feature attribution, counterfactuals, or natural language rationales)?
The research aims to determine if LLMs can act as a reliable 'judge' in the XAI ecosystem, providing a standardized way to benchmark how well models explain their decision-making processes.