Panos Parpas
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Panos Parpas has a total of 87 co-authors. Here are the top co-authors by number of shared publications:
Berç Rustem
Imperial College London, London, UK
18 shared publications- Edo Abraham
Department of Water Management, Delft University of Technology, 2628 CN Delft, The Netherlands
6 shared publications I. Stoianov
5 shared publications- Berç Rustem
Imperial College
4 shared publications - Chin Pang Ho
Department of Data Science, City University of Hong Kong, Hong Kong
4 shared publications - Grigorios A. Pavliotis
Imperial College London
4 shared publications
Latest Publications
- 13 August 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Scientific Computing
- Vol. 48 (5) , S232-S257
Multilevel Regularized Newton Methods with Fast Convergence Rates
We introduce new multilevel methods for solving large-scale unconstrained optimization problems. Specifically, the philosophy of multilevel methods is applied to Newton-type methods that regularize the Newton subproblem using second-order information from a coarse (low dimensional) subproblem. The new regularized multilevel methods provably converge from any initialization point and enjoy faster convergence rates than gradient descent. In particular, for arbitrary functions with Lipschitz continuous Hessians, we show that their convergence rate interpolates between the rate of gradient descent and that of the cubic Newton method. If, additionally, the objective function is assumed to be convex, then the proposed method converges with the fast [Formula: see text] rate. Hence, since the updates are generated using a coarse model in low dimensions, the theoretical results of this paper significantly speed up the convergence of Newton-type or preconditioned gradient methods in practical applications. Preliminary numerical results suggest that the proposed multilevel algorithms are significantly faster than current state-of-the-art methods.
- 13 August 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Scientific Computing
- Vol. 48 (5) , S232-S257
Multilevel Regularized Newton Methods with Fast Convergence Rates
We introduce new multilevel methods for solving large-scale unconstrained optimization problems. Specifically, the philosophy of multilevel methods is applied to Newton-type methods that regularize the Newton subproblem using second-order information from a coarse (low dimensional) subproblem. The new regularized multilevel methods provably converge from any initialization point and enjoy faster convergence rates than gradient descent. In particular, for arbitrary functions with Lipschitz continuous Hessians, we show that their convergence rate interpolates between the rate of gradient descent and that of the cubic Newton method. If, additionally, the objective function is assumed to be convex, then the proposed method converges with the fast [Formula: see text] rate. Hence, since the updates are generated using a coarse model in low dimensions, the theoretical results of this paper significantly speed up the convergence of Newton-type or preconditioned gradient methods in practical applications. Preliminary numerical results suggest that the proposed multilevel algorithms are significantly faster than current state-of-the-art methods.
- Open Access
Convergence rates of Newton's method for strongly self-concordant minimization
Newton's method has been thoroughly studied for the class of self-concordant functions. However, a local analysis specific to strongly self-concordant functions (a subclass of the former) is missing from the literature. The local quadratic rate of strongly self-concordant functions follows, of course, from the known results for self-concordant functions. However, it is not known whether strongly self-concordant functions enjoy better theoretical properties. In this paper, we study the local convergence of Newton's method for this subclass. We show that its quadratic convergence rate differs from that of general self-concordant functions. In particular, it is provably faster for a wide range of objective functions and benefits from a larger region of local convergence. Thus, the results of this paper close the gap in the theoretical understanding of Newton's method applied to strongly self-concordant functions.
Open AccessConvergence rates of Newton's method for strongly self-concordant minimization
Newton's method has been thoroughly studied for the class of self-concordant functions. However, a local analysis specific to strongly self-concordant functions (a subclass of the former) is missing from the literature. The local quadratic rate of strongly self-concordant functions follows, of course, from the known results for self-concordant functions. However, it is not known whether strongly self-concordant functions enjoy better theoretical properties. In this paper, we study the local convergence of Newton's method for this subclass. We show that its quadratic convergence rate differs from that of general self-concordant functions. In particular, it is provably faster for a wide range of objective functions and benefits from a larger region of local convergence. Thus, the results of this paper close the gap in the theoretical understanding of Newton's method applied to strongly self-concordant functions.
- Open Access
- 14 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 336-343
Stable Multilevel Deep Neural Networks for Option Pricing and xVAs Using Forward-Backward Stochastic Differential Equations
Deep learning techniques have significantly impacted fields such as image processing, computer vision, and natural language processing. However, their influence on quantitative finance, particularly in option pricing, hedging, and portfolio management, has been limited. Traditional financial applications rely on rigorous mathematical models and established numerical methods like Monte Carlo and finite difference methods. However, these methods struggle with high-dimensional problems. Recent works propose using Deep Neural Networks (DNNs) to solve high-dimensional Partial Differential Equations (PDEs) in finance, potentially overcoming the limitations of conventional techniques. Despite the recent progress, DNN methods face high computational costs, stability issues, and generally fail to meet the high accuracy requirements of the financial industry. This paper addresses these challenges. To address the high computational cost of training deep neural networks for financial applications, we propose a multilevel architecture inspired by multilevel Monte Carlo methods. To enhance stability, we adopt a dynamical systems perspective and utilize the NAIS-Net architecture, which ensures global asymptotic stability. For improving accuracy, we leverage the forward and backward systems of stochastic differential equations (FBSDEs) for option pricing problems. We also provide a theoretical argument that suggests that the proposed approach is superior to Physics Informed Neural Networks. Finally, the proposed methodology is implemented on several option pricing and xVA problems and is shown to achieve higher efficiency and accuracy, notably improving the pricing of options and xVAs by an order of magnitude over state-of-the-art methods.
- 14 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 336-343
Open AccessStable Multilevel Deep Neural Networks for Option Pricing and xVAs Using Forward-Backward Stochastic Differential Equations
Deep learning techniques have significantly impacted fields such as image processing, computer vision, and natural language processing. However, their influence on quantitative finance, particularly in option pricing, hedging, and portfolio management, has been limited. Traditional financial applications rely on rigorous mathematical models and established numerical methods like Monte Carlo and finite difference methods. However, these methods struggle with high-dimensional problems. Recent works propose using Deep Neural Networks (DNNs) to solve high-dimensional Partial Differential Equations (PDEs) in finance, potentially overcoming the limitations of conventional techniques. Despite the recent progress, DNN methods face high computational costs, stability issues, and generally fail to meet the high accuracy requirements of the financial industry. This paper addresses these challenges. To address the high computational cost of training deep neural networks for financial applications, we propose a multilevel architecture inspired by multilevel Monte Carlo methods. To enhance stability, we adopt a dynamical systems perspective and utilize the NAIS-Net architecture, which ensures global asymptotic stability. For improving accuracy, we leverage the forward and backward systems of stochastic differential equations (FBSDEs) for option pricing problems. We also provide a theoretical argument that suggests that the proposed approach is superior to Physics Informed Neural Networks. Finally, the proposed methodology is implemented on several option pricing and xVA problems and is shown to achieve higher efficiency and accuracy, notably improving the pricing of options and xVAs by an order of magnitude over state-of-the-art methods.
- Open Access
- 14 September 2024
- Published by Springer Nature in Journal of Optimization Theory and Applications
- Vol. 203 (3) , 2509-2559
A Multilevel Method for Self-Concordant Minimization
The analysis of second-order optimization methods based either on sub-sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates has only been shown to be scale-invariant only under specific assumptions on the problem structure. The second shortfall is that the fast convergence rates of second-order methods have only been established by making assumptions regarding the input data. In this paper, we propose a randomized Newton method for self-concordant functions to address both shortfalls. We propose a Self-concordant Iterative-minimization-Galerkin-based Multilevel Algorithm (SIGMA) and establish its super-linear convergence rate using the theory of self-concordant functions. Our analysis is based on the connections between multigrid optimization methods, and the role of coarse-grained or reduced-order models in the computation of search directions. We take advantage of the insights from the analysis to significantly improve the performance of second-order methods in machine learning applications. We report encouraging initial experiments that suggest SIGMA outperforms other state-of-the-art sub-sampled/sketched Newton methods for both medium and large-scale problems.
- 14 September 2024
- Published by Springer Nature in Journal of Optimization Theory and Applications
- Vol. 203 (3) , 2509-2559
Open AccessA Multilevel Method for Self-Concordant Minimization
The analysis of second-order optimization methods based either on sub-sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates has only been shown to be scale-invariant only under specific assumptions on the problem structure. The second shortfall is that the fast convergence rates of second-order methods have only been established by making assumptions regarding the input data. In this paper, we propose a randomized Newton method for self-concordant functions to address both shortfalls. We propose a Self-concordant Iterative-minimization-Galerkin-based Multilevel Algorithm (SIGMA) and establish its super-linear convergence rate using the theory of self-concordant functions. Our analysis is based on the connections between multigrid optimization methods, and the role of coarse-grained or reduced-order models in the computation of search directions. We take advantage of the insights from the analysis to significantly improve the performance of second-order methods in machine learning applications. We report encouraging initial experiments that suggest SIGMA outperforms other state-of-the-art sub-sampled/sketched Newton methods for both medium and large-scale problems.
- 5 August 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Dynamical Systems
- Vol. 23 (3) , 2208-2241
Stochastic Mirror Descent for Convex Optimization with Consensus Constraints
The mirror descent algorithm is known to be effective in situations where it is beneficial to adapt the mirror map to the underlying geometry of the optimization model. However, the effect of mirror maps on the geometry of distributed optimization problems has not been previously addressed. In this paper we study an exact distributed mirror descent algorithm in continuous time under additive noise. We establish a linear convergence rate of the proposed dynamics for the setting of convex optimization. Our analysis draws motivation from the augmented Lagrangian and its relation to gradient tracking. To further explore the benefits of mirror maps in a distributed setting we present a preconditioned variant of our algorithm with an additional mirror map over the Lagrangian dual variables. This allows our method to adapt to both the geometry of the primal variables and the geometry of the consensus constraint. We also propose a Gauss–Seidel type discretization scheme for the proposed method and establish its linear convergence rate. For certain classes of problems we identify mirror maps that mitigate the effect of the graph’s spectral properties on the convergence rate of the algorithm. Using numerical experiments, we demonstrate the efficiency of the methodology on convex models, both with and without constraints. Our findings show that the proposed method outperforms other methods, especially in scenarios where the model’s geometry is not captured by the standard Euclidean norm.
- 5 August 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Dynamical Systems
- Vol. 23 (3) , 2208-2241
Stochastic Mirror Descent for Convex Optimization with Consensus Constraints
The mirror descent algorithm is known to be effective in situations where it is beneficial to adapt the mirror map to the underlying geometry of the optimization model. However, the effect of mirror maps on the geometry of distributed optimization problems has not been previously addressed. In this paper we study an exact distributed mirror descent algorithm in continuous time under additive noise. We establish a linear convergence rate of the proposed dynamics for the setting of convex optimization. Our analysis draws motivation from the augmented Lagrangian and its relation to gradient tracking. To further explore the benefits of mirror maps in a distributed setting we present a preconditioned variant of our algorithm with an additional mirror map over the Lagrangian dual variables. This allows our method to adapt to both the geometry of the primal variables and the geometry of the consensus constraint. We also propose a Gauss–Seidel type discretization scheme for the proposed method and establish its linear convergence rate. For certain classes of problems we identify mirror maps that mitigate the effect of the graph’s spectral properties on the convergence rate of the algorithm. Using numerical experiments, we demonstrate the efficiency of the methodology on convex models, both with and without constraints. Our findings show that the proposed method outperforms other methods, especially in scenarios where the model’s geometry is not captured by the standard Euclidean norm.
- Open Access
Multilevel Regularized Newton Methods with Fast Convergence Rates
We introduce new multilevel methods for solving large-scale unconstrained optimization problems. Specifically, the philosophy of multilevel methods is applied to Newton-type methods that regularize the Newton sub-problem using second order information from a coarse (low dimensional) sub-problem. The new \emph{regularized multilevel methods} provably converge from any initialization point and enjoy faster convergence rates than Gradient Descent. In particular, for arbitrary functions with Lipschitz continuous Hessians, we show that their convergence rate interpolates between the rate of Gradient Descent and that of the cubic Newton method. If, additionally, the objective function is assumed to be convex, then the proposed method converges with the fast $\mathcal{O}(k^{-2})$ rate. Hence, since the updates are generated using a \emph{coarse} model in low dimensions, the theoretical results of this paper significantly speed-up the convergence of Newton-type or preconditioned gradient methods in practical applications. Preliminary numerical results suggest that the proposed multilevel algorithms are significantly faster than current state-of-the-art methods.Open AccessMultilevel Regularized Newton Methods with Fast Convergence Rates
We introduce new multilevel methods for solving large-scale unconstrained optimization problems. Specifically, the philosophy of multilevel methods is applied to Newton-type methods that regularize the Newton sub-problem using second order information from a coarse (low dimensional) sub-problem. The new \emph{regularized multilevel methods} provably converge from any initialization point and enjoy faster convergence rates than Gradient Descent. In particular, for arbitrary functions with Lipschitz continuous Hessians, we show that their convergence rate interpolates between the rate of Gradient Descent and that of the cubic Newton method. If, additionally, the objective function is assumed to be convex, then the proposed method converges with the fast $\mathcal{O}(k^{-2})$ rate. Hence, since the updates are generated using a \emph{coarse} model in low dimensions, the theoretical results of this paper significantly speed-up the convergence of Newton-type or preconditioned gradient methods in practical applications. Preliminary numerical results suggest that the proposed multilevel algorithms are significantly faster than current state-of-the-art methods. - Open Access
A Multilevel Method for Self-Concordant Minimization
The analysis of second-order optimization methods based either on sub-sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates has only been shown to be scale-invariant only under specific assumptions on the problem structure. The second shortfall is that the fast convergence rates of second-order methods have only been established by making assumptions regarding the input data. In this paper, we propose a randomized Newton method for self-concordant functions to address both shortfalls. We propose a Self-concordant Iterative-minimization-Galerkin-based Multilevel Algorithm (SIGMA) and establish its super-linear convergence rate using the theory of self-concordant functions. Our analysis is based on the connections between multigrid optimization methods, and the role of coarse-grained or reduced-order models in the computation of search directions. We take advantage of the insights from theanalysis to significantly improve the performance of second-order methods in machine learning applications. We report encouraging initial experiments that suggest SIGMA outperforms other state-of-the-art sub-sampled/sketched Newton methods for both medium and large-scale problems.
Open AccessA Multilevel Method for Self-Concordant Minimization
The analysis of second-order optimization methods based either on sub-sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates has only been shown to be scale-invariant only under specific assumptions on the problem structure. The second shortfall is that the fast convergence rates of second-order methods have only been established by making assumptions regarding the input data. In this paper, we propose a randomized Newton method for self-concordant functions to address both shortfalls. We propose a Self-concordant Iterative-minimization-Galerkin-based Multilevel Algorithm (SIGMA) and establish its super-linear convergence rate using the theory of self-concordant functions. Our analysis is based on the connections between multigrid optimization methods, and the role of coarse-grained or reduced-order models in the computation of search directions. We take advantage of the insights from theanalysis to significantly improve the performance of second-order methods in machine learning applications. We report encouraging initial experiments that suggest SIGMA outperforms other state-of-the-art sub-sampled/sketched Newton methods for both medium and large-scale problems.
- 6 March 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Scientific Computing
- Vol. 46 (2) , A770-A797
Using Witten Laplacians to Locate Index-1 Saddle Points
We introduce a new stochastic algorithm to locate the index-1 saddle points of a function [Formula: see text], with [Formula: see text] possibly large. This algorithm can be seen as an equivalent of the stochastic gradient descent which is a natural stochastic process to locate local minima. It relies on two ingredients: (i) the concentration properties on index-1 saddle points of the first eigenmodes of the Witten Laplacian (associated with [Formula: see text]) on 1-forms and (ii) a probabilistic representation of a partial differential equation involving this differential operator. Numerical examples on simple molecular systems illustrate the efficacy of the proposed approach. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available in https://github.com/pp500/Stochastic-Saddle-Point-Dynamics and in the supplementary materials ( Stochastic-Saddle-Point-Dynamics-main.zip [167KB]). [Formula: see text]
- 6 March 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Scientific Computing
- Vol. 46 (2) , A770-A797
Using Witten Laplacians to Locate Index-1 Saddle Points
We introduce a new stochastic algorithm to locate the index-1 saddle points of a function [Formula: see text], with [Formula: see text] possibly large. This algorithm can be seen as an equivalent of the stochastic gradient descent which is a natural stochastic process to locate local minima. It relies on two ingredients: (i) the concentration properties on index-1 saddle points of the first eigenmodes of the Witten Laplacian (associated with [Formula: see text]) on 1-forms and (ii) a probabilistic representation of a partial differential equation involving this differential operator. Numerical examples on simple molecular systems illustrate the efficacy of the proposed approach. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available in https://github.com/pp500/Stochastic-Saddle-Point-Dynamics and in the supplementary materials ( Stochastic-Saddle-Point-Dynamics-main.zip [167KB]). [Formula: see text]
- Open Access
Simba: A Scalable Bilevel Preconditioned Gradient Method for Fast Evasion of Flat Areas and Saddle Points
The convergence behaviour of first-order methods can be severely slowed down when applied to high-dimensional non-convex functions due to the presence of saddle points. If, additionally, the saddles are surrounded by large plateaus, it is highly likely that the first-order methods will converge to sub-optimal solutions. In machine learning applications, sub-optimal solutions mean poor generalization performance. They are also related to the issue of hyper-parameter tuning, since, in the pursuit of solutions that yield lower errors, a tremendous amount of time is required on selecting the hyper-parameters appropriately. A natural way to tackle the limitations of first-order methods is to employ the Hessian information. However, methods that incorporate the Hessian do not scale or, if they do, they are very slow for modern applications. Here, we propose Simba, a scalable preconditioned gradient method, to address the main limitations of the first-order methods. The method is very simple to implement. It maintains a single precondition matrix that it is constructed as the outer product of the moving average of the gradients. To significantly reduce the computational cost of forming and inverting the preconditioner, we draw links with the multilevel optimization methods. These links enables us to construct preconditioners in a randomized manner. Our numerical experiments verify the scalability of Simba as well as its efficacy near saddles and flat areas. Further, we demonstrate that Simba offers a satisfactory generalization performance on standard benchmark residual networks. We also analyze Simba and show its linear convergence rate for strongly convex functions.
Open AccessSimba: A Scalable Bilevel Preconditioned Gradient Method for Fast Evasion of Flat Areas and Saddle Points
The convergence behaviour of first-order methods can be severely slowed down when applied to high-dimensional non-convex functions due to the presence of saddle points. If, additionally, the saddles are surrounded by large plateaus, it is highly likely that the first-order methods will converge to sub-optimal solutions. In machine learning applications, sub-optimal solutions mean poor generalization performance. They are also related to the issue of hyper-parameter tuning, since, in the pursuit of solutions that yield lower errors, a tremendous amount of time is required on selecting the hyper-parameters appropriately. A natural way to tackle the limitations of first-order methods is to employ the Hessian information. However, methods that incorporate the Hessian do not scale or, if they do, they are very slow for modern applications. Here, we propose Simba, a scalable preconditioned gradient method, to address the main limitations of the first-order methods. The method is very simple to implement. It maintains a single precondition matrix that it is constructed as the outer product of the moving average of the gradients. To significantly reduce the computational cost of forming and inverting the preconditioner, we draw links with the multilevel optimization methods. These links enables us to construct preconditioners in a randomized manner. Our numerical experiments verify the scalability of Simba as well as its efficacy near saddles and flat areas. Further, we demonstrate that Simba offers a satisfactory generalization performance on standard benchmark residual networks. We also analyze Simba and show its linear convergence rate for strongly convex functions.
- Open Access
Using Witten Laplacians to locate index-1 saddle points
We introduce a new stochastic algorithm to locate the index-1 saddle points of a function $V:\mathbb R^d \to \mathbb R$, with $d$ possibly large. This algorithm can be seen as an equivalent of the stochastic gradient descent which is a natural stochastic process to locate local minima. It relies on two ingredients: (i) the concentration properties on index-1 saddle points of the first eigenmodes of the Witten Laplacian (associated with $V$) on $1$-forms and (ii) a probabilistic representation of a partial differential equation involving this differential operator. Numerical examples on simple molecular systems illustrate the efficacy of the proposed approach.Open AccessUsing Witten Laplacians to locate index-1 saddle points
We introduce a new stochastic algorithm to locate the index-1 saddle points of a function $V:\mathbb R^d \to \mathbb R$, with $d$ possibly large. This algorithm can be seen as an equivalent of the stochastic gradient descent which is a natural stochastic process to locate local minima. It relies on two ingredients: (i) the concentration properties on index-1 saddle points of the first eigenmodes of the Witten Laplacian (associated with $V$) on $1$-forms and (ii) a probabilistic representation of a partial differential equation involving this differential operator. Numerical examples on simple molecular systems illustrate the efficacy of the proposed approach.