In this article we propose a novel technique for the re-calibration of Machine Learning (ML) models. This technique is based on the computation of confidence intervals for the probability scores provided by any ML model. Compared to existing and commonly used calibration methods, the proposed approach has two important advantages: first, under weak assumptions it provides theoretical guarantees about calibration; second, this method does not require any further data other than the training set used for ML model development. We illustrate the effectiveness of the proposed approach on a benchmark dataset for COVID-19 diagnosis, by comparing the proposed method against commonly used re-calibration techniques.

Campagner, A., Famiglini, L., Cabitza, F. (2022). Re-calibrating Machine Learning Models Using Confidence Interval Bounds. In 19th International Conference on Modeling Decisions for Artificial Intelligence, MDAI 2022 (pp.132-142). Springer Science and Business Media Deutschland GmbH [10.1007/978-3-031-13448-7_11].

Re-calibrating Machine Learning Models Using Confidence Interval Bounds

Campagner A.
;
Famiglini L.;Cabitza F.
2022

Abstract

In this article we propose a novel technique for the re-calibration of Machine Learning (ML) models. This technique is based on the computation of confidence intervals for the probability scores provided by any ML model. Compared to existing and commonly used calibration methods, the proposed approach has two important advantages: first, under weak assumptions it provides theoretical guarantees about calibration; second, this method does not require any further data other than the training set used for ML model development. We illustrate the effectiveness of the proposed approach on a benchmark dataset for COVID-19 diagnosis, by comparing the proposed method against commonly used re-calibration techniques.
paper
Calibration; Confidence interval; Machine Learning; Trustable AI;
English
19th International Conference on Modeling Decisions for Artificial Intelligence, MDAI 2022 - 30 August 2022 through 2 September 2022
2022
19th International Conference on Modeling Decisions for Artificial Intelligence, MDAI 2022
9783031134470
2022
13408
132
142
none
Campagner, A., Famiglini, L., Cabitza, F. (2022). Re-calibrating Machine Learning Models Using Confidence Interval Bounds. In 19th International Conference on Modeling Decisions for Artificial Intelligence, MDAI 2022 (pp.132-142). Springer Science and Business Media Deutschland GmbH [10.1007/978-3-031-13448-7_11].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/394396
Citazioni
  • Scopus 1
  • ???jsp.display-item.citation.isi??? 1
Social impact