Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence
Open Access
- 10 April 2026
- journal article
- Published by IOP Publishing in Machine Learning: Earth
- Vol. 2 (1) , 015013
Abstract
Transfer learning (TL) is a powerful tool for enhancing the performance of neural networks (NNs) in applications such as weather and climate prediction and turbulence modeling. TL enables models to generalize to out-of-distribution data with minimal training data from the new system. In this study, we employ a 9-layer convolutional NN to predict the subgrid forcing in a two-layer ocean quasi-geostrophic system and examine which metrics best describe its performance and generalizability to unseen dynamical regimes. Fourier analysis of the NN kernels reveals that they learn low-pass, Gabor, and high-pass filters, regardless of whether the training data are isotropic or anisotropic. By analyzing the activation spectra, we identify why NNs fail to generalize without TL and how TL can overcome these limitations: the learned weights and biases from one dataset underestimate the out-of-distribution sample spectra as they pass through the network, leading to an underestimation of output spectra. By re-training only one layer with data from the target system, this underestimation is corrected, enabling the NN to produce predictions that match the target spectra. These findings are broadly applicable to data-driven parameterization of dynamical systems.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Backscatter in energetically-constrained Leith parameterizationsOcean Modelling, 2023
- Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LESPhysica D: Nonlinear Phenomena, 2022
- Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learningJournal of Computational Physics, 2022
- Transfer learning enhanced physics informed neural network for phase-field modeling of fractureTheoretical and Applied Fracture Mechanics, 2020
- Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High‐Resolution SimulationsGeophysical Research Letters, 2017
- Large‐eddy simulation of subtropical cloud‐topped boundary layers: 1. A forcing framework with closed surface energy balanceJournal of Advances in Modeling Earth Systems, 2016
- Using a resolution function to regulate parameterizations of oceanic mesoscale eddy effectsOcean Modelling, 2013
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989
- Computational Design of the Basic Dynamical Processes of the UCLA General Circulation Model1977
- Pseudospectral approximation to two-dimensional turbulenceJournal of Computational Physics, 1973