The emergence of black-box, subsymbolic, and statistical AI systems has motivated a rapid increase in the interest regarding explainable AI (XAI), which encompasses both inherently explainable techniques, as well as approaches to make black-box AI systems explainable to human decision makers. Rather than always making black boxes transparent, these approaches are at risk of painting the black boxes white, thus failing to provide a level of transparency that would increase the system’s usability and comprehensibility, or even at risk of generating new errors (i.e., white-box paradox). To address these usability-related issues, in this work we focus on the cognitive dimension of users’ perception of explanations and XAI systems. We investigated these perceptions in light of their relationship with users’ characteristics (e.g., expertise) through a questionnaire-based user study involved 44 cardiology residents and specialists in an AI-supported ECG reading task. Our results point to the relevance and correlation of the dimensions of trust, perceived quality of explanations, and tendency to defer the decision process to automation (i.e., technology dominance). This contribution calls for the evaluation of AI-based support systems from a human–AI interaction-oriented perspective, laying the ground for further investigation of XAI and its effects on decision making and user experience.

Cabitza, F., Campagner, A., Natali, C., Parimbelli, E., Ronzio, L., Cameli, M. (2023). Painting the Black Box White: Experimental Findings from Applying XAI to an ECG Reading Setting. MACHINE LEARNING AND KNOWLEDGE EXTRACTION, 5(1), 269-286 [10.3390/make5010017].

Painting the Black Box White: Experimental Findings from Applying XAI to an ECG Reading Setting

Cabitza F.
;
Campagner A.;Natali C.;
2023

Abstract

The emergence of black-box, subsymbolic, and statistical AI systems has motivated a rapid increase in the interest regarding explainable AI (XAI), which encompasses both inherently explainable techniques, as well as approaches to make black-box AI systems explainable to human decision makers. Rather than always making black boxes transparent, these approaches are at risk of painting the black boxes white, thus failing to provide a level of transparency that would increase the system’s usability and comprehensibility, or even at risk of generating new errors (i.e., white-box paradox). To address these usability-related issues, in this work we focus on the cognitive dimension of users’ perception of explanations and XAI systems. We investigated these perceptions in light of their relationship with users’ characteristics (e.g., expertise) through a questionnaire-based user study involved 44 cardiology residents and specialists in an AI-supported ECG reading task. Our results point to the relevance and correlation of the dimensions of trust, perceived quality of explanations, and tendency to defer the decision process to automation (i.e., technology dominance). This contribution calls for the evaluation of AI-based support systems from a human–AI interaction-oriented perspective, laying the ground for further investigation of XAI and its effects on decision making and user experience.
Articolo in rivista - Articolo scientifico
artificial intelligence; decision support systems; ECG; explainable AI; XAI;
English
8-mar-2023
2023
5
1
269
286
open
Cabitza, F., Campagner, A., Natali, C., Parimbelli, E., Ronzio, L., Cameli, M. (2023). Painting the Black Box White: Experimental Findings from Applying XAI to an ECG Reading Setting. MACHINE LEARNING AND KNOWLEDGE EXTRACTION, 5(1), 269-286 [10.3390/make5010017].
File in questo prodotto:
File Dimensione Formato  
Cabitza-2023-Mach Learn Knowledge Extract-VoR.pdf

accesso aperto

Descrizione: Article
Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Licenza: Creative Commons
Dimensione 3.48 MB
Formato Adobe PDF
3.48 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/433638
Citazioni
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 3
Social impact