Langevin dynamics based algorithm e-TH$\varepsilon$O POULA for stochastic optimization problems with discontinuous stochastic gradient
Preprint
Abstract
We introduce a new Langevin dynamics based algorithm, called e-TH$\varepsilon$O POULA, to solve optimization problems with discontinuous stochastic gradients which naturally appear in real-world applications such as quantile estimation, vector quantization, CVaR minimization, and regularized optimization problems involving ReLU neural networks. We demonstrate both theoretically and numerically the applicability of the e-TH$\varepsilon$O POULA algorithm. More precisely, under the conditions that the stochastic gradient is locally Lipschitz in average and satisfies a certain convexity at infinity condition, we establish non-asymptotic error bounds for e-TH$\varepsilon$O POULA in Wasserstein distances and provide a non-asymptotic estimate for the expected excess risk, which can be controlled to be arbitrarily small. Three key applications in finance and insurance are provided, namely, multi-period portfolio optimization, transfer learning in multi-period portfolio optimization, and insurance claim prediction, which involve neural networks with (Leaky)-ReLU activation functions. Numerical experiments conducted using real-world datasets illustrate the superior empirical performance of e-TH$\varepsilon$O POULA compared to SGLD, TUSLA, ADAM, and AMSGrad in terms of model accuracy.Keywords
All Related Versions
This publication has 39 references indexed in Scilit:
- Nonasymptotic Estimates for Stochastic Gradient Langevin Dynamics Under Local Conditions in Nonconvex OptimizationApplied Mathematics & Optimization, 2023
- Reinforcement learning and stochastic optimisationFinance and Stochastics, 2021
- Machine learning with kernels for portfolio valuation and risk managementFinance and Stochastics, 2021
- A hybrid deep learning method for optimal insurance strategies: Algorithms and convergence analysisInsurance: Mathematics and Economics, 2021
- Sampling from Non-smooth Distributions Through Langevin DiffusionMethodology and Computing in Applied Probability, 2020
- Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimizationOptimization Letters, 2020
- Bias regularization in neural network models for general insurance pricingEuropean Actuarial Journal, 2019
- Generalized linear models for dependent frequency and severity of insurance claimsInsurance: Mathematics and Economics, 2016
- Gradient boosting trees for auto insurance loss cost modeling and predictionExpert Systems with Applications, 2012
- Stochastic Control in Discrete and Continuous Time2009