A large family of black box methods rely on surrogates of the unknown, possibly non linear non convex reward function. While it is common to assume stationarity of the reward, many real-world problems satisfy this assumption only locally, hindering the spread application of such methods. This paper proposes a novel nonstationary regression model combining Decision Trees and Support Vector Machine (SVM) classification for a hierarchical non-axis-aligned partition of the input space. Gaussian Process (GP) regression is performed within each identified subregion. The resulting nonstationary regression model is the Treed Gaussian process with Support Vector Machine (SVMTGP), and we investigate the sampling efficiency from using our a model within a Bayesian optimization (BO) context. Empirically, we show how the resulting algorithm, SVMTGP-BO never underperforms BO when this is applied to an homogeneous Gaussian process, while it shows always better performance compared to the homogeneous model with nonlinear functions with complex landscapes.

Candelieri, A., Pedrielli, G. (2021). Treed-Gaussian Processes with Support Vector Machines as Nodes for Nonstationary Bayesian Optimization. In Proceedings - Winter Simulation Conference (pp.1-12). Institute of Electrical and Electronics Engineers Inc. [10.1109/WSC52266.2021.9715514].

Treed-Gaussian Processes with Support Vector Machines as Nodes for Nonstationary Bayesian Optimization

Candelieri A.;
2021

Abstract

A large family of black box methods rely on surrogates of the unknown, possibly non linear non convex reward function. While it is common to assume stationarity of the reward, many real-world problems satisfy this assumption only locally, hindering the spread application of such methods. This paper proposes a novel nonstationary regression model combining Decision Trees and Support Vector Machine (SVM) classification for a hierarchical non-axis-aligned partition of the input space. Gaussian Process (GP) regression is performed within each identified subregion. The resulting nonstationary regression model is the Treed Gaussian process with Support Vector Machine (SVMTGP), and we investigate the sampling efficiency from using our a model within a Bayesian optimization (BO) context. Empirically, we show how the resulting algorithm, SVMTGP-BO never underperforms BO when this is applied to an homogeneous Gaussian process, while it shows always better performance compared to the homogeneous model with nonlinear functions with complex landscapes.
Si
slide + paper
Bayesian Optimization, Support Vector Machines, Gaussian Processes, non-stationary functions
English
2021 Winter Simulation Conference, WSC 2021 - 12 December 2021 through 15 December 2021
9781665433112
Candelieri, A., Pedrielli, G. (2021). Treed-Gaussian Processes with Support Vector Machines as Nodes for Nonstationary Bayesian Optimization. In Proceedings - Winter Simulation Conference (pp.1-12). Institute of Electrical and Electronics Engineers Inc. [10.1109/WSC52266.2021.9715514].
Candelieri, A; Pedrielli, G
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/396708
Citazioni
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
Social impact