Most machine learning algorithms are configured by a set of hyperparameters whose values must be carefully chosen and which often considerably impact performance. To avoid a time-consuming and irreproducible manual process of trial-and-error to find well-performing hyperparameter configurations, various automatic hyperparameter optimization (HPO) methods—for example, based on resampling error estimation for supervised machine learning—can be employed. After introducing HPO from a general perspective, this paper reviews important HPO methods, from simple techniques such as grid or random search to more advanced methods like evolution strategies, Bayesian optimization, Hyperband, and racing. This work gives practical recommendations regarding important choices to be made when conducting HPO, including the HPO algorithms themselves, performance evaluation, how to combine HPO with machine learning pipelines, runtime improvements, and parallelization.

中文翻译：

---

超参数优化：基础、算法、最佳实践和开放挑战

大多数机器学习算法都是由一组超参数配置的，这些超参数的值必须仔细选择，并且通常会显着影响性能。为了避免耗时且不可重现的手动试错过程以找到性能良好的超参数配置，可以采用各种自动超参数优化 (HPO) 方法——例如，基于监督机器学习的重采样误差估计。从一般角度介绍 HPO 之后，本文回顾了重要的 HPO 方法，从网格或随机搜索等简单技术到进化策略、贝叶斯优化、Hyperband 和赛车等更高级的方法。这项工作给出了关于进行 HPO 时要做出的重要选择的实用建议，包括 HPO 算法本身，