Loading...

A data-driven robust optimization algorithm for black-box cases: An application to hyper-parameter optimization of machine learning algorithms

Seifi, F ; Sharif University of Technology | 2021

336 Viewed
  1. Type of Document: Article
  2. DOI: 10.1016/j.cie.2021.107581
  3. Publisher: Elsevier Ltd , 2021
  4. Abstract:
  5. The huge availability of data in the last decade has raised the opportunity for the better use of data in decision-making processes. The idea of using the existing data to achieve a more coherent reality solution has led to a branch of optimization called data-driven optimization. On the one hand, the presence of uncertain variables in these datasets makes it crucial to design robust optimization methods in this area. On the other hand, in many real-world problems, the closed-form of the objective function is not available and a meta-model based framework is necessary. Motivated by the above points, in this paper a Gaussian process is used in a Bayesian optimization framework to design a method that is consistent with the data in a predefined confidence level. The advantage of the proposed method is that it is computationally tractable in addition to being robust and independent of the objective function's form. As one of the applications of the proposed algorithm, hyper-parameter optimization for deep learning is investigated. The proposed method can help find the optimal hyper-parameters that are robust with respect to noise. © 2021 Elsevier Ltd
  6. Keywords:
  7. Decision making ; Deep learning ; Gaussian distribution ; Gaussian noise (electronic) ; Learning algorithms ; Parameter estimation ; Bayesian optimization ; Black-box optimization ; Data driven ; Data-driven optimization ; Gaussian Processes ; Hyper-parameter optimizations ; Hyper-parameter tuning ; Objective functions ; Robust optimization ; Optimization
  8. Source: Computers and Industrial Engineering ; Volume 160 , 2021 ; 03608352 (ISSN)
  9. URL: https://www.sciencedirect.com/science/article/abs/pii/S036083522100485X