Residual Multi-Fidelity Neural Network Computing
Preprint
Abstract
In this work, we consider the general problem of constructing a neural network surrogate model using multi-fidelity information. Motivated by rigorous error and complexity estimates for ReLU neural networks, given an inexpensive low-fidelity and an expensive high-fidelity computational model, we present a residual multi-fidelity computational framework that formulates the correlation between models as a residual function, a possibly non-linear mapping between 1) the shared input space of the models together with the low-fidelity model output and 2) the discrepancy between the two model outputs. To accomplish this, we train two neural networks to work in concert. The first network learns the residual function on a small set of high-fidelity and low-fidelity data. Once trained, this network is used to generate additional synthetic high-fidelity data, which is used in the training of a second network. This second network, once trained, acts as our surrogate for the high-fidelity quantity of interest. We present three numerical examples to demonstrate the power of the proposed framework. In particular, we show that dramatic savings in computational cost may be achieved when the output predictions are desired to be accurate within small tolerances.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- Multi-fidelity surrogate modeling using long short-term memory networksComputer Methods in Applied Mechanics and Engineering, 2022
- Multi-fidelity regression using artificial neural networks: Efficient approximation of parameter-dependent output quantitiesComputer Methods in Applied Mechanics and Engineering, 2022
- A Theoretical Analysis of Deep Neural Networks and Parametric PDEsConstructive Approximation, 2021
- A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equationsPartial Differential Equations and Applications, 2020
- The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational ProblemsCommunications in Mathematics and Statistics, 2018
- Neural network closures for nonlinear model order reductionAdvances in Computational Mathematics, 2018
- A Multi Level Monte Carlo method with control variate for elliptic PDEs with log-normal coefficientsStochastics and Partial Differential Equations: Analysis and Computations, 2015
- Multi-index Monte Carlo: when sparsity meets samplingNumerische Mathematik, 2015
- Multilevel Monte Carlo methods and applications to elliptic PDEs with random coefficientsComputing and Visualization in Science, 2011
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989