External validation of risk scores to predict in-hospital mortality in patients hospitalized due to coronavirus disease 2019

Background: The coronavirus disease 2019 (COVID-19) presents an urgent threat to global health. Prediction models that accurately estimate mortality risk in hospitalized patients could assist medical staff in treatment and allocating limited resources. Aims: To externally validate two promising previously published risk scores that predict in-hospital mortality among hospitalized COVID-19 patients. Methods: Two prospective cohorts were available; a cohort of 1028 patients admitted to one of nine hospitals in Lombardy, Italy (the Lombardy cohort) and a cohort of 432 patients admitted to a hospital in Leiden, the Netherlands (the Leiden cohort). The endpoint was in-hospital mortality. All patients were adult and tested COVID-19 PCR-positive. Model discrimination and calibration were assessed. Results: The C-statistic of the 4C mortality score was good in the Lombardy cohort (0.85, 95CI: 0.82−0.89) and in the Leiden cohort (0.87, 95CI: 0.80−0.94). Model calibration was acceptable in the Lombardy cohort but poor in the Leiden cohort due to the model systematically overpredicting the mortality risk for all patients. The C-statistic of the CURB-65 score was good in the Lombardy cohort (0.80, 95CI: 0.75−0.85) and in the Leiden cohort (0.82, 95CI: 0.76−0.88). The mortality rate in the CURB-65 development cohort was much lower than the mortality rate in the Lombardy cohort. A similar but less pronounced trend was found for patients in the Leiden cohort. Conclusion: Although performances did not differ greatly, the 4C mortality score showed the best performance. However, because of quickly changing circumstances, model recalibration may be necessary before using the 4C mortality score.