Neural Network Accelerated Implicit Filtering: Integrating Neural Network Surrogates With Provably Convergent Derivative Free Optimization Methods
Brian Irwin, Eldad Haber, Raviv Gal, Avi Ziv
In this paper, we introduce neural network accelerated implicit filtering (NNAIF), a novel family of methods for solving noisy derivative free (i.e. black box, zeroth order) optimization problems. NNAIF intelligently combines the established literature on implicit filtering (IF) optimization methods with a neural network (NN) surrogate model of the objective function, resulting in accelerated derivative free methods for unconstrained optimization problems. The NN surrogate model consists of a fixed number of parameters, which can be as few as $\approx 1.3 \times 10^{4}$, that are updated as NNAIF progresses. We show that NNAIF directly inherits the convergence properties of IF optimization methods, and thus NNAIF is guaranteed to converge towards a critical point of the objective function under appropriate assumptions. Numerical experiments with $31$ noisy problems from the CUTEst optimization benchmark set demonstrate the benefits and costs associated with NNAIF. These benefits include NNAIF’s ability to minimize structured functions of several thousand variables much more rapidly than well-known alternatives, such as Covariance Matrix Adaptation Evolution Strategy (CMA-ES) and finite difference based variants of gradient descent (GD) and BFGS, as well as its namesake IF.
Discussion
-
https://www.websitescrawl.com/domain-list-19900At SOBOBA Medical Weight Loss clinics, we have been helping patients successfully lose weight safely in Orange County, and Los Angeles County since 1968. Our physicians and staff focus solely on weight loss so we are able to focus on your success. 100% Medical Doctor Weight Loss Clinics. They prescribe ozempic, B-12 Injections, HCG Injections, B Complex Injections, Lipo-B Injections, Mounjaro
Reply