The Art and Science of Benchmarking Multi-objective Optimizers
Speaker: Lennart Schäpermeier
Abstract: In many practical optimization problems, multiple, at least in part contradictory, goals need to be balanced, for example, in package delivery (minimize costs, delivery times and emissions), machine learning (maximize model performance, minimize model complexity), or engineering (minimize production costs and weight of a given component). Multi-objective optimization algorithms support in finding trade-offs by computationally identifying efficient alternatives between all goals rather than producing a single "best" solution. However, performance evaluation and systematic development of different algorithmic ideas and approaches cannot be based on the anecdotal solution of a few real-world problems. Instead, we need systematic benchmarks that relate algorithm performance to problem features during evaluation. Only this enables practitioners to make informed algorithm selection decisions for their specific problem.
In this talk, we begin with a (quite visual) perspective on established test/benchmarking problems which reveals that current state-of-the-art procedures have severe shortcomings in several aspects. We'll then introduce a new structured benchmarking approach called BONO-Bench that addresses many of the issues for a large set of bi-objective problems.
Short Bio: Lennart Schäpermeier is a researcher in data science and optimization, focusing on multi-objective optimization, (automated) algorithm selection and benchmarking. He is a member of the Computational Social Science and Systems Analysis research group at the Department of Information Systems in Münster. Previously, he pursued his PhD research at the Chair of Big Data Analytics in Transportation at TU Dresden.