Understanding High-Dimensional Bayesian Optimization
Papenmeier, Leonard; Poloczek, Matthias; Nardi, Luigi
Zusammenfassung
Recent work reported that simple Bayesian optimization (BO) methods perform well for high-dimensional real-world tasks, seemingly contradicting prior work and tribal knowledge. This paper investigates why. We identify underlying challenges that arise in high-dimensional BO and explain why recent methods succeed. Our empirical analysis shows that vanishing gradients caused by Gaussian process (GP) initialization schemes play a major role in the failures of high-dimensional Bayesian optimization (HDBO) and that methods that promote local search behaviors are better suited for the task. We find that maximum likelihood estimation (MLE) of GP length scales suffices for state-of-the-art performance. Based on this, we propose a simple variant of MLE called MSR that leverages these findings to achieve state-of-the-art performance on a comprehensive set of real-world applications. We present targeted experiments to illustrate and confirm our findings.
Schlüsselwörter
Bayesian optimization; global optimization; Gaussian process; high-dimensional
Zitieren als
Papenmeier, L., Poloczek, M., & Nardi, L. (2025). Understanding High-Dimensional Bayesian Optimization. In Singh, A., Fazel, M., Hsu, D., Lacoste-Julien, S., Berkenkamp, F., Maharaj, T., Wagstaff, K., & Zhu, J. (Eds.),
Proceedings of Machine Learning Research (PMLR) (-, pp. 47902–47923). Proceedings of Machine Learning Research (PMLR): Vol. 267. Vancouver, Canada: MLResearchPress.
Details
Publikationstyp
Forschungsartikel in Sammelband (Konferenz)
Begutachtet
Ja
Publikationsstatus
Veröffentlicht
Jahr
2025
Konferenz
Proceedings of the Forty-Second International Conference on Machine Learning
Konferenzort
Vancouver Convention Center
Band
267
Buchtitel
Proceedings of Machine Learning Research (PMLR)
Herausgeber
Singh, Aarti; Fazel, Maryam; Hsu, Daniel; Lacoste-Julien, Simon; Berkenkamp, Felix; Maharaj, Tegan; Wagstaff, Kiri; Zhu, Jerry
Erste Seite
47902
Letzte Seite
47923
Auflage
-
Band
267
Reihe
Proceedings of Machine Learning Research (PMLR)
Verlag
MLResearchPress
Ort
Vancouver, Canada
Sprache
Englisch
ISSN
2640-3498
ISBN
-
DOI
Gesamter Text
