Page 171 - 《软件学报》2024年第4期

钱鸿 等: 基于动态批量评估的绿色无梯度优化方法 1749

[36] Li L, Jamieson K, Rostamizadeh A, et al. A system for massively parallel hyperparameter tuning. In: Proc. of the Machine
Learning and Systems 2020. 2020.
[37] Li S, Xing W, Kirby R, et al. Multi-fidelity Bayesian optimization via deep neural networks. In: Advances in Neural Information
Processing Systems 33. 2020.
[38] Belakaria S, Deshwal A, Doppa J. Multi-fidelity multi-objective Bayesian optimization: An output space entropy search approach.
In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. 2020. 10035−10043.
[39] Takeno S, Fukuoka H, Tsukada Y, et al. Takeuchi I, Karasuyama M. Multi-fidelity Bayesian optimization with max-value entropy
search and its parallelization. In: Proc. of the 37th Int’l Conf. on Machine Learning, 2020. 9334−9345.
[40] Roux N, Schmidt M, Bach F. A stochastic gradient method with an exponential convergence rate for finite training sets. In:
Advances in Neural Information Processing Systems 25. 2012. 2672−2680.
[41] Qian C. Distributed Pareto optimization for large-scale noisy subset selection. IEEE Trans. on Evolutionary Computation, 2020,
24(4): 694−707.
[42] Liu Y, Hu YQ, Qian H, et al. ZOOpt: A toolbox for derivative-free optimization. Science China Information Sciences, 2022,
65(10).
[43] Johnson R, Zhang T. Accelerating stochastic gradient descent using predictive variance reduction. In: Advances in Neural
Information Processing Systems 26. 2013. 315−323.
[44] Defazio A, Bach F, Lacoste-Julien S. SAGA: A fast incremental gradient method with support for non-strongly convex composite
objectives. In: Advances in Neural Information Processing Systems 27. 2014. 1646−1654.
[45] Hansen N, Müller S, Koumoutsakos P. Reducing the time complexity of the derandomized evolution strategy with covariance
matrix adaptation (CMA-ES). Evolutionary Computation, 2003, 11(1): 1−18.
[46] Bäck T, Schwefel H. An overview of evolutionary algorithms for parameter optimization. Evolutionary Computation, 1993, 1(1):
1−23.
[47] Jin Y, Wang H, Chugh T, et al. Data-driven evolutionary optimization: An overview and case studies. IEEE Trans. on Evolutionary
Computation, 2018, 23(2): 442−458.
[48] Shahriari B, Swersky K, Wang Z, et al. Taking the human out of the loop: A review of Bayesian optimization. Proc. of the IEEE,
2016, 104(1): 148−175.
[49] Garnett R. Bayesian Optimization. Cambridge University Press, 2023.
[50] Bartlett P, Gabillon V, Valko M. A simple parameter-free and adaptive approach to optimization under a minimal local smoothness
assumption. In: Proc. of the 2019 Algorithmic Learning Theory. 2019. 184−206.
[51] Munos R. From bandits to Monte-Carlo tree search: The optimistic principle applied to optimization and planning. Foundations and
Trends in Machine Learning, 2014, 7(1): 1−129.
[52] Valko M, Carpentier A, Munos R. Stochastic simultaneous optimistic optimization. In: Proc. of the 30th Int’l Conf. on Machine
Learning 2013. 2013. 19−27.
[53] Jones D, Perttunen C, Stuckman B. Lipschitzian optimization without the Lipschitz constant. Journal of Optimization Theory and
Applications, 1993, 79: 157−181.
[54] Boer P, Kroese D, Mannor S, et al. A tutorial on the cross-entropy method. Annals of Operations Research, 2005, 134(1).
[55] Hansen N, Ostermeier A. Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix
adaptation. In: Proc. of the 1996 IEEE Int’l Conf. on Evolutionary Computation. Nayoya University, 1996. 312−317.
[56] Hansen N, Ostermeier A. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 2001, 9(2):
159−195.
[57] Li L, Jamieson K, DeSalvo G, et al. Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of
Machine Learning Research, 2017, 18(185): 1−52.
[58] Lindauer M, Eggensperger K, Feurer M, et al. SMAC3: A versatile Bayesian optimization package for hyperparameter
optimization. Journal of Machine Learning Research, 2022, 23(54): 1−9.
[59] González J, Dai Z, Hennig P, et al. Batch Bayesian optimization via local penalization. In: Proc. of the 19th Int’l Conf. on
Artificial Intelligence and Statistics. 2016. 648−657.