In-Context Operator Learning with Prompts for Differential Equation Problems
Preprint
Abstract
This paper introduces a new neural-network-based approach, namely In-Context Operator Networks (ICON), to simultaneously learn operators from the prompted data and apply it to new questions during the inference stage, without any weight update. Existing methods are limited to using a neural network to approximate a specific equation solution or a specific operator, requiring retraining when switching to a new problem with different equations. By training a single neural network as an operator learner, we can not only get rid of retraining (even fine-tuning) the neural network for new problems, but also leverage the commonalities shared across operators so that only a few demos in the prompt are needed when learning a new operator. Our numerical results show the neural network's capability as a few-shot operator learner for a diversified type of differential equation problems, including forward and inverse problems of ordinary differential equations (ODEs), partial differential equations (PDEs), and mean-field control (MFC) problems, and also show that it can generalize its learning capability to operators beyond the training distribution.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Alternating the population and control neural networks to solve high-dimensional stochastic mean-field gamesProceedings of the National Academy of Sciences, 2021
- Learning nonlinear operators via DeepONet based on the universal approximation theorem of operatorsNature Machine Intelligence, 2021
- Weak adversarial networks for high-dimensional partial differential equationsJournal of Computational Physics, 2020
- A machine learning framework for solving high-dimensional mean field game and mean field control problemsProceedings of the National Academy of Sciences, 2020
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equationsJournal of Computational Physics, 2019
- DGM: A deep learning algorithm for solving partial differential equationsJournal of Computational Physics, 2018
- Solving high-dimensional partial differential equations using deep learningProceedings of the National Academy of Sciences, 2018
- The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational ProblemsCommunications in Mathematics and Statistics, 2018