In this paper, we contribute to the deconstruction of the concept of accuracy with respect to machine learning systems that are used in human decision making, and specifically in medicine. We argue that, by taking a socio-technical stance, it is necessary to move from the idea that these systems are “agents that can err”, to the idea that these are just tools by which humans can interpret new cases in light of the technologically-mediated interpretation of past cases, like if they were wearing a pair of tinted glasses. In this new narrative, accuracy is a meaningless construct, while it is important that beholders can “believe in their eyes” (or spectacles), and therefore trust the tool enough to make sensible decisions.

Cabitza, F., Campagner, A., Datteri, E. (2021). To Err is (only) Human. Reflections on How to Move from Accuracy to Trust for Medical AI. In Exploring Innovation in a Digital World: Cultural and Organizational Challenges (pp.36-49). Cham : Springer Science and Business Media Deutschland GmbH [10.1007/978-3-030-87842-9_4].

To Err is (only) Human. Reflections on How to Move from Accuracy to Trust for Medical AI

Cabitza, F;Campagner, A;Datteri, E
2021

Abstract

In this paper, we contribute to the deconstruction of the concept of accuracy with respect to machine learning systems that are used in human decision making, and specifically in medicine. We argue that, by taking a socio-technical stance, it is necessary to move from the idea that these systems are “agents that can err”, to the idea that these are just tools by which humans can interpret new cases in light of the technologically-mediated interpretation of past cases, like if they were wearing a pair of tinted glasses. In this new narrative, accuracy is a meaningless construct, while it is important that beholders can “believe in their eyes” (or spectacles), and therefore trust the tool enough to make sensible decisions.
paper
Accuracy; Decision support systems; Machine learning; Medical artificial intelligence
English
Annual conference of the Italian Chapter of AIS, 2020
2020
Ceci, F; Prencipe, A; Spagnoletti, P
Exploring Innovation in a Digital World: Cultural and Organizational Challenges
978-3-030-87841-2
2021
51
36
49
open
Cabitza, F., Campagner, A., Datteri, E. (2021). To Err is (only) Human. Reflections on How to Move from Accuracy to Trust for Medical AI. In Exploring Innovation in a Digital World: Cultural and Organizational Challenges (pp.36-49). Cham : Springer Science and Business Media Deutschland GmbH [10.1007/978-3-030-87842-9_4].
File in questo prodotto:
File Dimensione Formato  
ItAIS2020_paper_7.pdf

accesso aperto

Descrizione: Conference Proceedings
Tipologia di allegato: Submitted Version (Pre-print)
Dimensione 568.1 kB
Formato Adobe PDF
568.1 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/388636
Citazioni
  • Scopus 7
  • ???jsp.display-item.citation.isi??? ND
Social impact