Prognostic models applied in medicine must be validated on independent samples, before their use can be recommended. The assessment of calibration, i.e., the model's ability to provide reliable predictions, is crucial in external validation studies. Besides having several shortcomings, statistical techniques such as the computation of the standardized mortality ratio (SMR) and its confidence intervals, the Hosmer-Lemeshow statistics, and the Cox calibration test, are all non-informative with respect to calibration across risk classes. Accordingly, calibration plots reporting expected versus observed outcomes across risk subsets have been used for many years. Erroneously, the points in the plot (frequently representing deciles of risk) have been connected with lines, generating false calibration curves. Here we propose a methodology to create a confidence band for the calibration curve based on a function that relates expected to observed probabilities across classes of risk. The calibration belt allows the ranges of risk to be spotted where there is a significant deviation from the ideal calibration, and the direction of the deviation to be indicated. This method thus offers a more analytical view in the assessment of quality of care, compared to other approaches. © 2011 Finazzi et al.

Finazzi, S., Poole, D., Luciani, D., Cogo, P., Bertolini, G. (2011). Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLOS ONE, 6(2), e16110 [10.1371/journal.pone.0016110].

Calibration belt for quality-of-care assessment based on dichotomous outcomes

LUCIANI, DAVIDE;
2011

Abstract

Prognostic models applied in medicine must be validated on independent samples, before their use can be recommended. The assessment of calibration, i.e., the model's ability to provide reliable predictions, is crucial in external validation studies. Besides having several shortcomings, statistical techniques such as the computation of the standardized mortality ratio (SMR) and its confidence intervals, the Hosmer-Lemeshow statistics, and the Cox calibration test, are all non-informative with respect to calibration across risk classes. Accordingly, calibration plots reporting expected versus observed outcomes across risk subsets have been used for many years. Erroneously, the points in the plot (frequently representing deciles of risk) have been connected with lines, generating false calibration curves. Here we propose a methodology to create a confidence band for the calibration curve based on a function that relates expected to observed probabilities across classes of risk. The calibration belt allows the ranges of risk to be spotted where there is a significant deviation from the ideal calibration, and the direction of the deviation to be indicated. This method thus offers a more analytical view in the assessment of quality of care, compared to other approaches. © 2011 Finazzi et al.
Articolo in rivista - Articolo scientifico
Calibration; Hospital Mortality; Humans; Intensive Care; Intensive Care Units; Models, Theoretical; Outcome Assessment (Health Care); Probability; Quality Assurance, Health Care; ROC Curve; Reference Values; Risk Assessment; Treatment Outcome; Quality of Health Care; Agricultural and Biological Sciences (all); Biochemistry, Genetics and Molecular Biology (all); Medicine (all)
English
2011
6
2
e16110
e16110
none
Finazzi, S., Poole, D., Luciani, D., Cogo, P., Bertolini, G. (2011). Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLOS ONE, 6(2), e16110 [10.1371/journal.pone.0016110].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/78261
Citazioni
  • Scopus 94
  • ???jsp.display-item.citation.isi??? 89
Social impact