A hierarchical expected improvement method for Bayesian optimization
Prof. C. F. Jeff Wu
Coca-Cola Chair in Engineering Statistics
H. Milton Stewart School of Industrial and Systems Engineering
Georgia Institute of Technology
X.Q. Deng Presidential Chair Professor
School of Data Science
The Chinese University of Hong Kong, Shenzhen
The Expected Improvement (EI) method is a widely-used Bayesian optimization method, which makes use of a fitted Gaussian process model for efficient black-box optimization. However, one key drawback of EI is that it is overly greedy in exploiting the fitted Gaussian process model, which results in suboptimal solutions. We propose a new hierarchical EI (HEI) framework, which makes use of a hierarchical Gaussian process model. HEI preserves a closed-form acquisition function, and corrects the over-greediness of EI by encouraging exploration. Under certain prior specifications, we prove the global convergence of HEI over a broad function space, and derive global convergence rates under smoothness assumptions on the objective function. We then introduce hyperparameter estimation methods which allow HEI to mimic a fully Bayesian procedure while avoiding expensive Markov-chain Monte Carlo sampling. Numerical experiments and a toy semiconductor optimization application show the improvement of HEI over existing black-box optimization methods.
(Authors: Zhehui Chen, Simon Mak, and C. F. Jeff Wu; in JASA T&M)
