KSKonstantinos Spiliopoulos

Konstantinos Spiliopoulos

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Konstantinos Spiliopoulos has a total of 45 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Transport map unadjusted Langevin algorithms: Learning and discretizing perturbed samplers

    by Benjamin J. Zhang, Youssef M. Marzouk, Konstantinos Spiliopoulos

    Langevin dynamics are widely used in sampling high-dimensional, non-Gaussian distributions whose densities are known up to a normalizing constant. In particular, there is strong interest in unadjusted Langevin algorithms (ULA), which directly discretize Langevin dynamics to estimate expectations over the target distribution. We study the use of transport maps that approximately normalize a target distribution as a way to precondition and accelerate the convergence of Langevin dynamics. We show that in continuous time, when a transport map is applied to Langevin dynamics, the result is a Riemannian manifold Langevin dynamics (RMLD) with metric defined by the transport map. We also show that applying a transport map to an irreversibly-perturbed ULA results in a geometry-informed irreversible perturbation (GiIrr) of the original dynamics. These connections suggest more systematic ways of learning metrics and perturbations, and also yield alternative discretizations of the RMLD described by the map, which we study. Under appropriate conditions, these discretized processes can be endowed with non-asymptotic bounds describing convergence to the target distribution in 2-Wasserstein distance. Illustrative numerical results complement our theoretical claims.


  • Open Access

    Importance sampling for stochastic reaction-diffusion equations in the moderate deviation regime

    by Ioannis Gasteratos, Michael Salins, Konstantinos Spiliopoulos

    We develop a provably efficient importance sampling scheme that estimates exit probabilities of solutions to small-noise stochastic reaction-diffusion equations from scaled neighborhoods of a stable equilibrium. The moderate deviation scaling allows for a local approximation of the nonlinear dynamics by their linearized version. In addition, we identify a finite-dimensional subspace where exits take place with high probability. Using stochastic control and variational methods we show that our scheme performs well both in the zero noise limit and pre-asymptotically. Simulation studies for stochastically perturbed bistable dynamics illustrate the theoretical results.

  • Open Access

    PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence

    by Justin Sirignano, Jonathan MacArt, Konstantinos Spiliopoulos

    Recent research has used deep learning to develop partial differential equation (PDE) models in science and engineering. The functional form of the PDE is determined by a neural network, and the neural network parameters are calibrated to available data. Calibration of the embedded neural network can be performed by optimizing over the PDE. Motivated by these applications, we rigorously study the optimization of a class of linear elliptic PDEs with neural network terms. The neural network parameters in the PDE are optimized using gradient descent, where the gradient is evaluated using an adjoint PDE. As the number of parameters become large, the PDE and adjoint PDE converge to a non-local PDE system. Using this limit PDE system, we are able to prove convergence of the neural network-PDE to a global minimum during the optimization. Finally, we use this adjoint method to train a neural network model for an application in fluid mechanics, in which the neural network functions as a closure model for the Reynolds-averaged Navier--Stokes (RANS) equations. The RANS neural network model is trained on several datasets for turbulent channel flow and is evaluated out-of-sample at different Reynolds numbers.

  • Open Access

    Fluctuation analysis for particle-based stochastic reaction-diffusion models

    by Max Heldman, Samuel Isaacson, Jingwei Ma, Konstantinos Spiliopoulos

    Recent works have derived and proven the large-population mean-field limit for several classes of particle-based stochastic reaction-diffusion (PBSRD) models. These limits correspond to systems of partial integral-differential equations (PIDEs) that generalize standard mass-action reaction-diffusion PDE models. In this work we derive and prove the next order fluctuation corrections to such limits, which we show satisfy systems of stochastic PIDEs with Gaussian noise. Numerical examples are presented to illustrate how including the fluctuation corrections can enable the accurate estimation of higher order statistics of the underlying PBSRD model.

  • Open Access

    On the large-time behaviour of affine Volterra processes

    by Antoine Jacquier, Alexandre Pannier, Konstantinos Spiliopoulos

    We show the existence of a stationary measure for a class of multidimensional stochastic Volterra systems of affine type. These processes are in general not Markovian, a shortcoming which hinders their large-time analysis. We circumvent this issue by lifting the system to a measure-valued stochastic PDE introduced by Cuchiero and Teichmann, whence we retrieve the Markov property. Leveraging on the associated generalised Feller property, we extend the Krylov-Bogoliubov theorem to this infinite-dimensional setting and thus establish an approach to the existence of invariant measures. We present concrete examples, including the rough Heston model from Mathematical Finance.

  • Open Access

    Moderate deviation principle for multiscale systems driven by fractional Brownian motion

    by Solesne Bourguin, Thanh Dang, Konstantinos Spiliopoulos

    In this paper we study the moderate deviations principle (MDP) for slow-fast stochastic dynamical systems where the slow motion is governed by small fractional Brownian motion (fBm) with Hurst parameter $H\in(1/2,1)$. We derive conditions on the moderate deviations scaling and on the Hurst parameter $H$ under which the MDP holds. In addition, we show that in typical situations the resulting action functional is discontinuous in $H$ at $H=1/2$, suggesting that the tail behavior of stochastic dynamical systems perturbed by fBm can have different characteristics than the tail behavior of such systems that are perturbed by standard Brownian motion.

  • Normalization effects on deep neural networks

    by Jiahui Yu, Konstantinos Spiliopoulos

    We study the effect of normalization on the layers of deep neural networks of feed-forward type. A given layer $ i $ with $ N_{i} $ hidden units is allowed to be normalized by $ 1/N_{i}^{\gamma_{i}} $ with $ \gamma_{i}\in[1/2,1] $ and we study the effect of the choice of the $ \gamma_{i} $ on the statistical behavior of the neural network's output (such as variance) as well as on the test accuracy on the MNIST data set. We find that in terms of variance of the neural network's output and test accuracy the best choice is to choose the $ \gamma_{i} $'s to be equal to one, which is the mean-field scaling. We also find that this is particularly true for the outer layer, in that the neural network's behavior is more sensitive in the scaling of the outer layer as opposed to the scaling of the inner layers. The mechanism for the mathematical analysis is an asymptotic expansion for the neural network's output. An important practical consequence of the analysis is that it provides a systematic and mathematically informed way to choose the learning rate hyperparameters. Such a choice guarantees that the neural network behaves in a statistically robust way as the $ N_i $ grow to infinity.


  • Open Access

    Normalization effects on shallow neural networks and related asymptotic expansions

    by Jiahui Yu, Konstantinos Spiliopoulos

    We consider shallow (single hidden layer) neural networks and characterize their performance when trained with stochastic gradient descent as the number of hidden units $N$ and gradient descent steps grow to infinity. In particular, we investigate the effect of different scaling schemes, which lead to different normalizations of the neural network, on the network's statistical output, closing the gap between the $1/\sqrt{N}$ and the mean-field $1/N$ normalization. We develop an asymptotic expansion for the neural network's statistical output pointwise with respect to the scaling parameter as the number of hidden units grows to infinity. Based on this expansion, we demonstrate mathematically that to leading order in $N$, there is no bias-variance trade off, in that both bias and variance (both explicitly characterized) decrease as the number of hidden units increases and time grows. In addition, we show that to leading order in $N$, the variance of the neural network's statistical output decays as the implied normalization by the scaling parameter approaches the mean field normalization. Numerical studies on the MNIST and CIFAR10 datasets show that test and train accuracy monotonically improve as the neural network's normalization gets closer to the mean field normalization.