The introduction of Artificial Intelligence as a seamless support to diagnostic decision-making in medicine holds promise to revolutionise clinical work and increase both efficiency as well as patient outcomes. Yet improvements in model accuracy alone do not guarantee clinical benefit. Empirical studies show that introducing AI into clinical workflows can unintentionally weaken the conditions that sustain expert judgement: it may narrow attention, suppress the evaluation of alternative hypotheses, and encourage premature acceptance of algorithmic suggestions. Over time, such interaction patterns risk generating inappropriate reliance, reduced vigilance, inhibition of skill acquisition and erosion of diagnostic competence. This thesis argues that the central challenge is not to improve predictive performance alone, but to actively shape how humans and AI systems interact as to sustain diagnostic reasoning and preserve professional agency--recognising that this may require rethinking long-standing doctrines that equate good system design with seamlessness, efficiency, and cognitive ease. To this end, the thesis develops and advances Frictional AI as a principled and actionable design paradigm that introduces structured cognitive friction—such as hypothesis-first sequencing, forced justification, or delayed disclosure—to support reflective engagement and counter automation bias. First, the thesis presents an interaction protocol perspective and beyond-accuracy evaluation to capture how AI systems influence reliance across levels of expertise, and consolidates the state of the evidence on AI-induced deskilling and upskilling inhibition in the medical literature. Second, through controlled studies, it show that explanation-centred strategies often fail to correct reliance, discussing a user study on the impact of misleading explanations. Third, it offers a conceptual synthesis of frictional AI as design strategies, reporting the results from the organisation of an international workshop series on the topic as the starting point for scholarly consolidation and consensus on the topic. This synthesis culminates in the Open–Multiple–Adjunct model, a high-level framework further validated through interviews with radiologists that relates friction-based interventions across both the model/data layer (e.g., exposing uncertainty, contestation, provenance) and the interface/protocol layer (e.g., hypothesis-first sequencing, commitment constraints, comparative views). Finally, it presents empirical evaluations of frictional interaction mechanisms in radiology, demonstrating that structured cognitive friction can reduce inappropriate reliance without unacceptable performance penalties and are viewed by practitioners as aligning with the temporal and interpretive logic of expert diagnostic reasoning. Finally, empirical evaluations of frictional interaction mechanisms in radiology show that structured cognitive friction can reduce inappropriate reliance without unacceptable performance penalties, and is experienced by practitioners as aligned with the temporal and interpretive logic of diagnostic work. The thesis contributes a conceptual foundation, evaluative methodology, and empirical evidence supporting frictional design as a viable path toward responsible and durable human–AI integration in clinical decision-making, reframing friction as a resource for cognitive engagement and skill sustainment.
The introduction of Artificial Intelligence as a seamless support to diagnostic decision-making in medicine holds promise to revolutionise clinical work and increase both efficiency as well as patient outcomes. Yet improvements in model accuracy alone do not guarantee clinical benefit. Empirical studies show that introducing AI into clinical workflows can unintentionally weaken the conditions that sustain expert judgement: it may narrow attention, suppress the evaluation of alternative hypotheses, and encourage premature acceptance of algorithmic suggestions. Over time, such interaction patterns risk generating inappropriate reliance, reduced vigilance, inhibition of skill acquisition and erosion of diagnostic competence. This thesis argues that the central challenge is not to improve predictive performance alone, but to actively shape how humans and AI systems interact as to sustain diagnostic reasoning and preserve professional agency--recognising that this may require rethinking long-standing doctrines that equate good system design with seamlessness, efficiency, and cognitive ease. To this end, the thesis develops and advances Frictional AI as a principled and actionable design paradigm that introduces structured cognitive friction—such as hypothesis-first sequencing, forced justification, or delayed disclosure—to support reflective engagement and counter automation bias. First, the thesis presents an interaction protocol perspective and beyond-accuracy evaluation to capture how AI systems influence reliance across levels of expertise, and consolidates the state of the evidence on AI-induced deskilling and upskilling inhibition in the medical literature. Second, through controlled studies, it show that explanation-centred strategies often fail to correct reliance, discussing a user study on the impact of misleading explanations. Third, it offers a conceptual synthesis of frictional AI as design strategies, reporting the results from the organisation of an international workshop series on the topic as the starting point for scholarly consolidation and consensus on the topic. This synthesis culminates in the Open–Multiple–Adjunct model, a high-level framework further validated through interviews with radiologists that relates friction-based interventions across both the model/data layer (e.g., exposing uncertainty, contestation, provenance) and the interface/protocol layer (e.g., hypothesis-first sequencing, commitment constraints, comparative views). Finally, it presents empirical evaluations of frictional interaction mechanisms in radiology, demonstrating that structured cognitive friction can reduce inappropriate reliance without unacceptable performance penalties and are viewed by practitioners as aligning with the temporal and interpretive logic of expert diagnostic reasoning. Finally, empirical evaluations of frictional interaction mechanisms in radiology show that structured cognitive friction can reduce inappropriate reliance without unacceptable performance penalties, and is experienced by practitioners as aligned with the temporal and interpretive logic of diagnostic work. The thesis contributes a conceptual foundation, evaluative methodology, and empirical evidence supporting frictional design as a viable path toward responsible and durable human–AI integration in clinical decision-making, reframing friction as a resource for cognitive engagement and skill sustainment.
Natali, C (2026). Frictional AI. Countering Over-Reliance on Clinical Decision Support Systems via Desirable Inefficiencies. (Tesi di dottorato, , 2026).
Frictional AI. Countering Over-Reliance on Clinical Decision Support Systems via Desirable Inefficiencies
NATALI, CHIARA
2026
Abstract
The introduction of Artificial Intelligence as a seamless support to diagnostic decision-making in medicine holds promise to revolutionise clinical work and increase both efficiency as well as patient outcomes. Yet improvements in model accuracy alone do not guarantee clinical benefit. Empirical studies show that introducing AI into clinical workflows can unintentionally weaken the conditions that sustain expert judgement: it may narrow attention, suppress the evaluation of alternative hypotheses, and encourage premature acceptance of algorithmic suggestions. Over time, such interaction patterns risk generating inappropriate reliance, reduced vigilance, inhibition of skill acquisition and erosion of diagnostic competence. This thesis argues that the central challenge is not to improve predictive performance alone, but to actively shape how humans and AI systems interact as to sustain diagnostic reasoning and preserve professional agency--recognising that this may require rethinking long-standing doctrines that equate good system design with seamlessness, efficiency, and cognitive ease. To this end, the thesis develops and advances Frictional AI as a principled and actionable design paradigm that introduces structured cognitive friction—such as hypothesis-first sequencing, forced justification, or delayed disclosure—to support reflective engagement and counter automation bias. First, the thesis presents an interaction protocol perspective and beyond-accuracy evaluation to capture how AI systems influence reliance across levels of expertise, and consolidates the state of the evidence on AI-induced deskilling and upskilling inhibition in the medical literature. Second, through controlled studies, it show that explanation-centred strategies often fail to correct reliance, discussing a user study on the impact of misleading explanations. Third, it offers a conceptual synthesis of frictional AI as design strategies, reporting the results from the organisation of an international workshop series on the topic as the starting point for scholarly consolidation and consensus on the topic. This synthesis culminates in the Open–Multiple–Adjunct model, a high-level framework further validated through interviews with radiologists that relates friction-based interventions across both the model/data layer (e.g., exposing uncertainty, contestation, provenance) and the interface/protocol layer (e.g., hypothesis-first sequencing, commitment constraints, comparative views). Finally, it presents empirical evaluations of frictional interaction mechanisms in radiology, demonstrating that structured cognitive friction can reduce inappropriate reliance without unacceptable performance penalties and are viewed by practitioners as aligning with the temporal and interpretive logic of expert diagnostic reasoning. Finally, empirical evaluations of frictional interaction mechanisms in radiology show that structured cognitive friction can reduce inappropriate reliance without unacceptable performance penalties, and is experienced by practitioners as aligned with the temporal and interpretive logic of diagnostic work. The thesis contributes a conceptual foundation, evaluative methodology, and empirical evidence supporting frictional design as a viable path toward responsible and durable human–AI integration in clinical decision-making, reframing friction as a resource for cognitive engagement and skill sustainment.| File | Dimensione | Formato | |
|---|---|---|---|
|
phd_unimib_897224.pdf
accesso aperto
Descrizione: Tesi definitiva revisionata
Tipologia di allegato:
Doctoral thesis
Dimensione
2.55 MB
Formato
Adobe PDF
|
2.55 MB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.


