Learning relevant contextual variables within Bayesian optimization

Julien Martinelli, Ayush Bharti, Armi Tiihonen, Louis Filstroff, S. T. John, Sabina J. Sloman, Patrick Rinke, Samuel Kaski

Keywords: Contextual Bayesian Optimization, Variable Selection, Gaussian Processes

TL;DR: We propose a variable selection method for identifying relevant contextual variables in Bayesian Optimization

Abstract:

Contextual Bayesian Optimization (CBO) efficiently optimizes black-box, expensive-to-
evaluate functions with respect to design variables, while simultaneously integrating relevant contextual information regarding the environment, such as experimental conditions.
However, the relevance of contextual variables is not necessarily known beforehand. Moreover, contextual variables can sometimes be optimized themselves, an overlooked setting by
current CBO algorithms. Optimizing contextual variables may be costly, which raises the
question of determining a minimal relevant subset. We address this problem using a novel
method, Sensitivity-Analysis-Driven Contextual BO (SADCBO). We learn the relevance of
context variables by sensitivity analysis of the posterior surrogate model, whilst minimizing the cost of optimization by leveraging recent developments on early stopping for BO.
We empirically evaluate our proposed SADCBO against alternatives on both synthetic and
real-world experiments, and demonstrate a consistent improvement across examples.

Submission Number: 59