Machine learning–based decision support systems (DSS) are attracting the interest of the medical community. Their usage, however, could have deep consequences in terms of biasing the doctor’s interpretation of a case through automation bias and deskilling. In this work we address the design of DSS with the goal of minimizing these biases through the design and implementation of programmed inefficiencies (PIs), that is, features with the stated purpose of making the reliance of the human doctor on the DSS less obvious (or more difficult). We illustrate this concept by presenting a real-life medical DSS, called DataWise, embedding different PIs and currently undergoing iterative prototyping and testing with the medical users in two clinical settings. We describe the main features of DataWise, and show how different PIs have been conveyed by prompting doctors for multiple input and using qualitative visualizations instead of precise, but possibly misleading, indications. Finally, we discuss the implications of this design approach to naturalistic decision making, especially in life-saving domains like medicine is.

Cabitza, F., Campagner, A., Ciucci, D., Seveso, A. (2019). Programmed Inefficiencies in DSS-Supported Human Decision Making. In International Conference on Modeling Decisions for Artificial Intelligence (MDAI 2019) (pp.201-212). Springer Verlag [10.1007/978-3-030-26773-5_18].

Programmed Inefficiencies in DSS-Supported Human Decision Making

Cabitza F.
;
Campagner A.;Ciucci D.;Seveso A.
2019

Abstract

Machine learning–based decision support systems (DSS) are attracting the interest of the medical community. Their usage, however, could have deep consequences in terms of biasing the doctor’s interpretation of a case through automation bias and deskilling. In this work we address the design of DSS with the goal of minimizing these biases through the design and implementation of programmed inefficiencies (PIs), that is, features with the stated purpose of making the reliance of the human doctor on the DSS less obvious (or more difficult). We illustrate this concept by presenting a real-life medical DSS, called DataWise, embedding different PIs and currently undergoing iterative prototyping and testing with the medical users in two clinical settings. We describe the main features of DataWise, and show how different PIs have been conveyed by prompting doctors for multiple input and using qualitative visualizations instead of precise, but possibly misleading, indications. Finally, we discuss the implications of this design approach to naturalistic decision making, especially in life-saving domains like medicine is.
No
paper
DSS; Human decision making; Uncertainty
English
16th International Conference on Modeling Decisions for Artificial Intelligence, MDAI 2019
9783030267728
Cabitza, F., Campagner, A., Ciucci, D., Seveso, A. (2019). Programmed Inefficiencies in DSS-Supported Human Decision Making. In International Conference on Modeling Decisions for Artificial Intelligence (MDAI 2019) (pp.201-212). Springer Verlag [10.1007/978-3-030-26773-5_18].
Cabitza, F; Campagner, A; Ciucci, D; Seveso, A
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/265893
Citazioni
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 2
Social impact