Document Type
Article
Department or Administrative Unit
Computer Science
Publication Date
4-2019
Abstract
We introduce an improved version of Random Search (RS), used here for hyperparameter optimization of machine learning algorithms. Unlike the standard RS, which generates for each trial new values for all hyperparameters, we generate new values for each hyperparameter with a probability of change. The intuition behind our approach is that a value that already triggered a good result is a good candidate for the next step, and should be tested in new combinations of hyperparameter values. Within the same computational budget, our method yields better results than the standard RS. Our theoretical results prove this statement. We test our method on a variation of one of the most commonly used objective function for this class of problems (the Grievank function) and for the hyperparameter optimization of a deep learning CNN architecture. Our results can be generalized to any optimization problem defined on a discrete domain.
Recommended Citation
Florea, Adrian-Cǎtǎlin and Andonie, Rǎzvan, "Weighted Random Search for Hyperparameter Optimization" (2019). All Faculty Scholarship for the College of the Sciences. 110.
https://digitalcommons.cwu.edu/cotsfac/110
Journal
International Journal of Computers, Communications and Control
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Rights
Copyright © 2019 Adrian-Catalin Florea, Razvan Andonie
Comments
This article was originally published in International Journal of Computers, Communications and Control. The full-text article from the publisher can be found here.