In-context operator learning with data prompts for differential equation problems
Open Access
- 19 September 2023
- journal article
- research article
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 120 (39)
Abstract
This paper introduces the paradigm of “in-context operator learning” and the corresponding model “In-Context Operator Networks” to simultaneously learn operators from the prompted data and apply it to new questions during the inference stage, without any weight update. Existing methods are limited to using a neural network to approximate a specific equation solution or a specific operator, requiring retraining when switching to a new problem with different equations. By training a single neural network as an operator learner, rather than a solution/operator approximator, we can not only get rid of retraining (even fine-tuning) the neural network for new problems but also leverage the commonalities shared across operators so that only a few examples in the prompt are needed when learning a new operator. Our numerical results show the capability of a single neural network as a few-shot operator learner for a diversified type of differential equation problems, including forward and inverse problems of ordinary differential equations, partial differential equations, and mean-field control problems, and also show that it can generalize its learning capability to operators beyond the training distribution.Keywords
All Related Versions
Funding Information
- DOD | USAF | AMC | Air Force Office of Scientific Research (MURI FA9550-18-502)
- DOD | USN | Office of Naval Research (N00014-18-1-2527)
- DOD | USN | Office of Naval Research (N00014-20-1-2093)
- DOD | USN | Office of Naval Research (N00014-20-1-2787)
This publication has 37 references indexed in Scilit:
- Transfer learning enhanced physics informed neural network for phase-field modeling of fractureTheoretical and Applied Fracture Mechanics, 2020
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equationsJournal of Computational Physics, 2019
- DGM: A deep learning algorithm for solving partial differential equationsJournal of Computational Physics, 2018
- Solving high-dimensional partial differential equations using deep learningProceedings of the National Academy of Sciences, 2018
- Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantificationJournal of Computational Physics, 2018
- The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational ProblemsCommunications in Mathematics and Statistics, 2018
- Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential EquationsCommunications in Mathematics and Statistics, 2017
- Deep Residual Learning for Image Recognition2016
- Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networksIEEE Transactions on Neural Networks, 1995
- Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systemsIEEE Transactions on Neural Networks, 1995