Searching for the optimal values of hyperparameters of a Machine Learning algorithm can be an extremely computationally expensive and awfully energivorous process. This paper proposes multi information source optimization as a strategy to drastically reduce computational time and energy consumed by this search process, where the information sources, with different computational costs, are smaller portions of a large dataset. The algorithm fits into the scheme of Gaussian Process based Bayesian optimization: an Augmented Gaussian Process is proposed, which is trained using only 'reliable' information among available sources. A novel acquisition function is also defined according to the Augmented Gaussian Process. Results on the hyperparameter optimization of a Support Vector Machine for a classification task on a large dataset are presented, with a comparison in terms of the misclassification error, on 10 fold-cross validation, and computational cost between the proposed approach and traditional hyperparameter optimization using the entire dataset.

Candelieri, A., Archetti, F., Ponti, A., Perego, R. (2020). Energy Efficient Hyperparameters Tuning through Augmented Gaussian Processes and Multi-information Source optimization. In 2020 7th International Conference on Soft Computing and Machine Intelligence, ISCMI 2020 (pp.34-38). Institute of Electrical and Electronics Engineers Inc. [10.1109/ISCMI51676.2020.9311599].

Energy Efficient Hyperparameters Tuning through Augmented Gaussian Processes and Multi-information Source optimization

Candelieri A.
Primo
;
Archetti F.
Secondo
;
Ponti A.;Perego R.
2020

Abstract

Searching for the optimal values of hyperparameters of a Machine Learning algorithm can be an extremely computationally expensive and awfully energivorous process. This paper proposes multi information source optimization as a strategy to drastically reduce computational time and energy consumed by this search process, where the information sources, with different computational costs, are smaller portions of a large dataset. The algorithm fits into the scheme of Gaussian Process based Bayesian optimization: an Augmented Gaussian Process is proposed, which is trained using only 'reliable' information among available sources. A novel acquisition function is also defined according to the Augmented Gaussian Process. Results on the hyperparameter optimization of a Support Vector Machine for a classification task on a large dataset are presented, with a comparison in terms of the misclassification error, on 10 fold-cross validation, and computational cost between the proposed approach and traditional hyperparameter optimization using the entire dataset.
paper
bayesian optimization; gaussian processes; green AI; green machine learning; multi information source optimization;
English
7th International Conference on Soft Computing and Machine Intelligence, ISCMI 2020 - NOV 14-15
2020
2020 7th International Conference on Soft Computing and Machine Intelligence, ISCMI 2020
978-1-7281-7559-1
2020
34
38
9311599
none
Candelieri, A., Archetti, F., Ponti, A., Perego, R. (2020). Energy Efficient Hyperparameters Tuning through Augmented Gaussian Processes and Multi-information Source optimization. In 2020 7th International Conference on Soft Computing and Machine Intelligence, ISCMI 2020 (pp.34-38). Institute of Electrical and Electronics Engineers Inc. [10.1109/ISCMI51676.2020.9311599].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/303236
Citazioni
  • Scopus 1
  • ???jsp.display-item.citation.isi??? 1
Social impact