High-dimensional model-based optimization based on noisy evaluations of computer games

Preuss M., Wagner T., Ginsbourger D.

Abstract
Most publications on surrogate models have focused either on the prediction quality or on the optimization performance. It is still unclear whether the prediction quality is indeed related to the suitability for optimization. Moreover, most of these studies only employ low-dimensional test cases. There are no results for popular surrogate models, such as kriging, for high-dimensional (n > 10) noisy problems. In this paper, we analyze both aspects by comparing different surrogate models on the noisy 22-dimensional car setup optimization problem, based on both, prediction quality and optimization performance. In order not to favor specific properties of the model, we run two conceptually different modern optimization methods on the surrogate models, CMA-ES and BOBYQA. It appears that kriging and random forests are very good modeling techniques with respect to both, prediction quality and suitability for optimization algorithms. © 2012 Springer-Verlag.

Keywords
Computer Games; Design and Analysis of Computer Experiments; Kriging; Model-Based Optimization; Sequential Parameter Optimization; The Open Racing Car Simulator