Global optimization, especially Bayesian optimization, has become the tool of choice in hyperparameter tuning and algorithmic configuration to optimize the generalization capability of machine learning algorithms. The contribution of this paper was to extend this approach to a complex algorithmic pipeline for predictive analytics, based on time-series clustering and artificial neural networks. The software environment R has been used with mlrMBO, a comprehensive and flexible toolbox for sequential model-based optimization. Random forest has been adopted as surrogate model, due to the nature of decision variables (i.e., conditional and discrete hyperparameters) of the case studies considered. Two acquisition functions have been considered: Expected improvement and lower confidence bound, and results are compared. The computational results, on a benchmark and a real-world dataset, show that even in a complex search space, up to 80 dimensions related to integer, categorical, and conditional variables (i.e., hyperparameters), sequential model-based optimization is an effective solution, with lower confidence bound requiring a lower number of function evaluations than expected improvement to find the same optimal solution.

Candelieri, A., Archetti, F. (2019). Global optimization in machine learning: the design of a predictive analytics application. SOFT COMPUTING, 23(9), 2969-2977 [10.1007/s00500-018-3597-8].

Global optimization in machine learning: the design of a predictive analytics application

Candelieri, Antonio
;
Archetti, Francesco
2019

Abstract

Global optimization, especially Bayesian optimization, has become the tool of choice in hyperparameter tuning and algorithmic configuration to optimize the generalization capability of machine learning algorithms. The contribution of this paper was to extend this approach to a complex algorithmic pipeline for predictive analytics, based on time-series clustering and artificial neural networks. The software environment R has been used with mlrMBO, a comprehensive and flexible toolbox for sequential model-based optimization. Random forest has been adopted as surrogate model, due to the nature of decision variables (i.e., conditional and discrete hyperparameters) of the case studies considered. Two acquisition functions have been considered: Expected improvement and lower confidence bound, and results are compared. The computational results, on a benchmark and a real-world dataset, show that even in a complex search space, up to 80 dimensions related to integer, categorical, and conditional variables (i.e., hyperparameters), sequential model-based optimization is an effective solution, with lower confidence bound requiring a lower number of function evaluations than expected improvement to find the same optimal solution.
Articolo in rivista - Articolo scientifico
Global optimization; Hyperparameters optimization; Machine learning;
hyperparameters optimization, global optimization, machine learning
English
1-nov-2018
2019
23
9
2969
2977
partially_open
Candelieri, A., Archetti, F. (2019). Global optimization in machine learning: the design of a predictive analytics application. SOFT COMPUTING, 23(9), 2969-2977 [10.1007/s00500-018-3597-8].
File in questo prodotto:
File Dimensione Formato  
Candelieri-2019-Soft Comput-AAM.pdf

accesso aperto

Descrizione: Article
Tipologia di allegato: Author’s Accepted Manuscript, AAM (Post-print)
Licenza: Altro
Dimensione 400.64 kB
Formato Adobe PDF
400.64 kB Adobe PDF Visualizza/Apri
Candelieri-2019-Soft Comput-VoR.pdf

Solo gestori archivio

Descrizione: Article
Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Licenza: Tutti i diritti riservati
Dimensione 692.85 kB
Formato Adobe PDF
692.85 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/226615
Citazioni
  • Scopus 15
  • ???jsp.display-item.citation.isi??? 13
Social impact