Giovanni Fantuzzi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Giovanni Fantuzzi has a total of 45 co-authors. Here are the top co-authors by number of shared publications:
- Andrew Wynn
Imperial College London
15 shared publications Andrew Wynn
Department of Aeronautics, Imperial College London, SW7 2AZ London UK
10 shared publications- Yang Zheng
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA, USA
8 shared publications Antonis Papachristodoulou
University of Oxford
7 shared publications- David Goluskin
University of Victoria
7 shared publications - Antonis Papachristodoulou
Department of Engineering Science, University of Oxford, Parks Road, Oxford OX1 3PJ, United Kingdom
6 shared publications
Latest Publications
- Open Access
- 11 May 2026
- Published by Springer Nature in Mathematical Foundations of Machine Learning
- Vol. 2 (1) , 2
Exact sequence interpolation with transformers
We prove that transformers can exactly interpolate datasets of finite input sequences in $$\mathbb {R}^d$$ , $$d\ge 2$$ , with corresponding output sequences of smaller or equal length. Specifically, given N sequences of arbitrary but finite lengths in $$\mathbb {R}^d$$ and output sequences of lengths $$m^1, \dots , m^N \in \mathbb {N}$$ , we construct a transformer with $$\mathcal {O}(\sum _{j=1}^N m^j)$$ blocks and $$\smash {\mathcal {O}(d \sum _{j=1}^N m^j)}$$ parameters that exactly interpolates the dataset. Our construction provides complexity estimates that are independent of the input sequence length, by alternating feed-forward and self-attention layers and by capitalizing on the clustering effect inherent to the latter. Our novel constructive method also uses low-rank parameter matrices in the self-attention mechanism, a common feature of practical transformer implementations. These results are first established in the hardmax self-attention setting, where the geometric structure permits an explicit and quantitative analysis, and are then extended to the softmax setting. Finally, we demonstrate the applicability of our exact interpolation construction to learning problems, in particular by providing convergence guarantees to a global minimizer under regularized training strategies. Our analysis contributes to the theoretical understanding of transformer models, offering an explanation for their excellent performance in exact sequence-to-sequence interpolation tasks.- 11 May 2026
- Published by Springer Nature in Mathematical Foundations of Machine Learning
- Vol. 2 (1) , 2
Open AccessExact sequence interpolation with transformers
We prove that transformers can exactly interpolate datasets of finite input sequences in $$\mathbb {R}^d$$ , $$d\ge 2$$ , with corresponding output sequences of smaller or equal length. Specifically, given N sequences of arbitrary but finite lengths in $$\mathbb {R}^d$$ and output sequences of lengths $$m^1, \dots , m^N \in \mathbb {N}$$ , we construct a transformer with $$\mathcal {O}(\sum _{j=1}^N m^j)$$ blocks and $$\smash {\mathcal {O}(d \sum _{j=1}^N m^j)}$$ parameters that exactly interpolates the dataset. Our construction provides complexity estimates that are independent of the input sequence length, by alternating feed-forward and self-attention layers and by capitalizing on the clustering effect inherent to the latter. Our novel constructive method also uses low-rank parameter matrices in the self-attention mechanism, a common feature of practical transformer implementations. These results are first established in the hardmax self-attention setting, where the geometric structure permits an explicit and quantitative analysis, and are then extended to the softmax setting. Finally, we demonstrate the applicability of our exact interpolation construction to learning problems, in particular by providing convergence guarantees to a global minimizer under regularized training strategies. Our analysis contributes to the theoretical understanding of transformer models, offering an explanation for their excellent performance in exact sequence-to-sequence interpolation tasks. - Open Access
Moment-SOS hierarchies for arrow-type polynomial matrix inequalities with applications to structural optimization
The Arrow Decomposition (AD) technique, initially introduced in [Mathematical Programming 190(1-2) (2021), pp 105-134], demonstrated superior scalability over the classical chordal decomposition in the context of Linear Matrix Inequalities (LMIs) if the matrix in question satisfied suitable assumptions. The primary objective of this paper is to extend the AD method to address Polynomial Optimization Problems (POPs) involving large-scale Polynomial Matrix Inequalities (PMIs), with the solution framework relying on moment-sum of square (mSOS) hierarchies. As a first step, we revisit the LMI case and weaken the conditions necessary for the key AD theorem presented in [Mathematical Programming 190(1-2) (2021), pp 105-134]. This modification allows the method to be applied to a broader range of problems. Next, we propose a practical procedure that reduces the number of additional variables, drawing on physical interpretations often found in structural optimization applications. For the PMI case, we explore two distinct approaches to combine the AD technique with mSOS hierarchies. One approach involves applying AD to the original POP before implementing the mSOS relaxation. The other approach applies AD directly to the mSOS relaxations of the POP. We establish convergence guarantees for both approaches and prove that theoretical properties extend to the polynomial case. Finally, we illustrate the significant computational advantages offered by the application of AD, particularly in the context of structural optimization problems.
Open AccessMoment-SOS hierarchies for arrow-type polynomial matrix inequalities with applications to structural optimization
The Arrow Decomposition (AD) technique, initially introduced in [Mathematical Programming 190(1-2) (2021), pp 105-134], demonstrated superior scalability over the classical chordal decomposition in the context of Linear Matrix Inequalities (LMIs) if the matrix in question satisfied suitable assumptions. The primary objective of this paper is to extend the AD method to address Polynomial Optimization Problems (POPs) involving large-scale Polynomial Matrix Inequalities (PMIs), with the solution framework relying on moment-sum of square (mSOS) hierarchies. As a first step, we revisit the LMI case and weaken the conditions necessary for the key AD theorem presented in [Mathematical Programming 190(1-2) (2021), pp 105-134]. This modification allows the method to be applied to a broader range of problems. Next, we propose a practical procedure that reduces the number of additional variables, drawing on physical interpretations often found in structural optimization applications. For the PMI case, we explore two distinct approaches to combine the AD technique with mSOS hierarchies. One approach involves applying AD to the original POP before implementing the mSOS relaxation. The other approach applies AD directly to the mSOS relaxations of the POP. We establish convergence guarantees for both approaches and prove that theoretical properties extend to the polynomial case. Finally, we illustrate the significant computational advantages offered by the application of AD, particularly in the context of structural optimization problems.
- 18 August 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Mathematics of Data Science
- Vol. 7 (3) , 1367-1393
Clustering in Pure-Attention Hardmax Transformers and Its Role in Sentiment Analysis
Transformers are extremely successful machine learning models whose mathematical properties remain poorly understood. Here, we rigorously characterize the behavior of transformers with hardmax self-attention and normalization sublayers as the number of layers tends to infinity. By viewing such transformers as discrete-time dynamical systems describing the evolution of points in a Euclidean space, and thanks to a geometric interpretation of the self-attention mechanism based on hyperplane separation, we show that the transformer inputs asymptotically converge to a clustered equilibrium determined by special points called leaders. We then leverage this theoretical understanding to solve sentiment analysis problems from language processing using a fully interpretable transformer model, which effectively captures “context” by clustering meaningless words around leader words carrying the most meaning. Finally, we outline remaining challenges to bridge the gap between the mathematical analysis of transformers and their real-life implementation.
- 18 August 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Mathematics of Data Science
- Vol. 7 (3) , 1367-1393
Clustering in Pure-Attention Hardmax Transformers and Its Role in Sentiment Analysis
Transformers are extremely successful machine learning models whose mathematical properties remain poorly understood. Here, we rigorously characterize the behavior of transformers with hardmax self-attention and normalization sublayers as the number of layers tends to infinity. By viewing such transformers as discrete-time dynamical systems describing the evolution of points in a Euclidean space, and thanks to a geometric interpretation of the self-attention mechanism based on hyperplane separation, we show that the transformer inputs asymptotically converge to a clustered equilibrium determined by special points called leaders. We then leverage this theoretical understanding to solve sentiment analysis problems from language processing using a fully interpretable transformer model, which effectively captures “context” by clustering meaningless words around leader words carrying the most meaning. Finally, we outline remaining challenges to bridge the gap between the mathematical analysis of transformers and their real-life implementation.
- Open Access
- 25 August 2024
- Published by Cambridge University Press (CUP) in Journal of Fluid Mechanics
- Vol. 993, A5
Internal heating profiles for which downward conduction is impossible
We consider an internally heated fluid between parallel plates with fixed thermal fluxes. For a large class of heat sources that vary in the direction of gravity, we prove that $\smash { \smash {{\langle {\delta T} \rangle _h}} } \geq \sigma R^{-1/3} - \mu$ , where $\smash { \smash {{\langle {\delta T} \rangle _h}} }$ is the average temperature difference between the bottom and top plates, $R$ is a ‘flux’ Rayleigh number and the constants $\sigma,\mu >0$ depend on the geometric properties of the internal heating. This result implies that mean downward conduction (for which $\smash { \smash {{\langle {\delta T} \rangle _h}} }< 0$ ) is impossible for a range of Rayleigh numbers smaller than a critical value $R_0:=(\sigma /\mu )^{3}$ . The bound demonstrates that $R_0$ depends on the heating distribution and can be made arbitrarily large by concentrating the heating near the bottom plate. However, for any given fixed heating profile of the class we consider, the corresponding value of $R_0$ is always finite. This points to a fundamental difference between internally heated convection and its limiting case of Rayleigh–Bénard convection with fixed-flux boundary conditions, for which $\smash {{\langle {\delta T} \rangle _h}}$ is known to be positive for all $R$ .- 25 August 2024
- Published by Cambridge University Press (CUP) in Journal of Fluid Mechanics
- Vol. 993, A5
Open AccessInternal heating profiles for which downward conduction is impossible
We consider an internally heated fluid between parallel plates with fixed thermal fluxes. For a large class of heat sources that vary in the direction of gravity, we prove that $\smash { \smash {{\langle {\delta T} \rangle _h}} } \geq \sigma R^{-1/3} - \mu$ , where $\smash { \smash {{\langle {\delta T} \rangle _h}} }$ is the average temperature difference between the bottom and top plates, $R$ is a ‘flux’ Rayleigh number and the constants $\sigma,\mu >0$ depend on the geometric properties of the internal heating. This result implies that mean downward conduction (for which $\smash { \smash {{\langle {\delta T} \rangle _h}} }< 0$ ) is impossible for a range of Rayleigh numbers smaller than a critical value $R_0:=(\sigma /\mu )^{3}$ . The bound demonstrates that $R_0$ depends on the heating distribution and can be made arbitrarily large by concentrating the heating near the bottom plate. However, for any given fixed heating profile of the class we consider, the corresponding value of $R_0$ is always finite. This points to a fundamental difference between internally heated convection and its limiting case of Rayleigh–Bénard convection with fixed-flux boundary conditions, for which $\smash {{\langle {\delta T} \rangle _h}}$ is known to be positive for all $R$ . - 1 July 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Scientific Computing
- Vol. 46 (4) , A2123-A2149
Global Minimization of Polynomial Integral Functionals
We describe a “discretize-then-relax” strategy to globally minimize integral functionals over functions [Formula: see text] in a Sobolev space subject to Dirichlet boundary conditions. The strategy applies whenever the integral functional depends polynomially on [Formula: see text] and its derivatives, even if it is nonconvex. The “discretize” step uses a bounded finite element scheme to approximate the integral minimization problem with a convergent hierarchy of polynomial optimization problems over a compact feasible set, indexed by the decreasing size [Formula: see text] of the finite element mesh. The “relax” step employs sparse moment-sum-of-squares relaxations to approximate each polynomial optimization problem with a hierarchy of convex semidefinite programs, indexed by an increasing relaxation order [Formula: see text]. We prove that, as [Formula: see text] and [Formula: see text], solutions of such semidefinite programs provide approximate minimizers that converge in a suitable sense (including in certain [Formula: see text] norms) to the global minimizer of the original integral functional if it is unique. We also report computational experiments showing that our numerical strategy works well even when technical conditions required by our theoretical analysis are not satisfied.
- 1 July 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Scientific Computing
- Vol. 46 (4) , A2123-A2149
Global Minimization of Polynomial Integral Functionals
We describe a “discretize-then-relax” strategy to globally minimize integral functionals over functions [Formula: see text] in a Sobolev space subject to Dirichlet boundary conditions. The strategy applies whenever the integral functional depends polynomially on [Formula: see text] and its derivatives, even if it is nonconvex. The “discretize” step uses a bounded finite element scheme to approximate the integral minimization problem with a convergent hierarchy of polynomial optimization problems over a compact feasible set, indexed by the decreasing size [Formula: see text] of the finite element mesh. The “relax” step employs sparse moment-sum-of-squares relaxations to approximate each polynomial optimization problem with a hierarchy of convex semidefinite programs, indexed by an increasing relaxation order [Formula: see text]. We prove that, as [Formula: see text] and [Formula: see text], solutions of such semidefinite programs provide approximate minimizers that converge in a suitable sense (including in certain [Formula: see text] norms) to the global minimizer of the original integral functional if it is unique. We also report computational experiments showing that our numerical strategy works well even when technical conditions required by our theoretical analysis are not satisfied.
- Open Access
Sharpness and non-sharpness of occupation measure bounds for integral variational problems
We analyze two recently proposed methods to establish a priori lower bounds on the minimum of general integral variational problems. The methods, which involve either `occupation measures' or a `pointwise dual relaxation' procedure, are shown to produce the same lower bound under a coercivity hypothesis ensuring their strong duality. We then show by a minimax argument that the methods actually evaluate the minimum for classes of one-dimensional, scalar-valued, or convex multidimensional problems. For generic problems, however, these methods should fail to capture the minimum and produce non-sharp lower bounds. We demonstrate this using two examples, the first of which is one-dimensional and scalar-valued with a non-convex constraint, and the second of which is multidimensional and non-convex in a different way. The latter example emphasizes the existence in multiple dimensions of nonlinear constraints on gradient fields that are ignored by occupation measures, but are built into the finer theory of gradient Young measures.
Open AccessSharpness and non-sharpness of occupation measure bounds for integral variational problems
We analyze two recently proposed methods to establish a priori lower bounds on the minimum of general integral variational problems. The methods, which involve either `occupation measures' or a `pointwise dual relaxation' procedure, are shown to produce the same lower bound under a coercivity hypothesis ensuring their strong duality. We then show by a minimax argument that the methods actually evaluate the minimum for classes of one-dimensional, scalar-valued, or convex multidimensional problems. For generic problems, however, these methods should fail to capture the minimum and produce non-sharp lower bounds. We demonstrate this using two examples, the first of which is one-dimensional and scalar-valued with a non-convex constraint, and the second of which is multidimensional and non-convex in a different way. The latter example emphasizes the existence in multiple dimensions of nonlinear constraints on gradient fields that are ignored by occupation measures, but are built into the finer theory of gradient Young measures.
- 27 March 2024
- Published by The Royal Society in Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Vol. 480 (2286)
Data-driven discovery of invariant measures
Invariant measures encode the long-time behaviour of a dynamical system. In this work, we propose an optimization-based method to discover invariant measures directly from data gathered from a system. Our method does not require an explicit model for the dynamics and allows one to target specific invariant measures, such as physical and ergodic measures. Moreover, it applies to both deterministic and stochastic dynamics in either continuous or discrete time. We provide convergence results and illustrate the performance of our method on data from the logistic map and a stochastic double-well system, for which invariant measures can be found by other means. We then use our method to approximate the physical measure of the chaotic attractor of the Rössler system, and we extract unstable periodic orbits embedded in this attractor by identifying discrete-time periodic points of a suitably defined Poincaré map. This final example is truly data-driven and shows that our method can significantly outperform previous approaches based on model identification.
- 27 March 2024
- Published by The Royal Society in Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Vol. 480 (2286)
Data-driven discovery of invariant measures
Invariant measures encode the long-time behaviour of a dynamical system. In this work, we propose an optimization-based method to discover invariant measures directly from data gathered from a system. Our method does not require an explicit model for the dynamics and allows one to target specific invariant measures, such as physical and ergodic measures. Moreover, it applies to both deterministic and stochastic dynamics in either continuous or discrete time. We provide convergence results and illustrate the performance of our method on data from the logistic map and a stochastic double-well system, for which invariant measures can be found by other means. We then use our method to approximate the physical measure of the chaotic attractor of the Rössler system, and we extract unstable periodic orbits embedded in this attractor by identifying discrete-time periodic points of a suitably defined Poincaré map. This final example is truly data-driven and shows that our method can significantly outperform previous approaches based on model identification.
- Open Access
Global minimization of polynomial integral functionals
We describe a `discretize-then-relax' strategy to globally minimize integral functionals over functions $u$ in a Sobolev space subject to Dirichlet boundary conditions. The strategy applies whenever the integral functional depends polynomially on $u$ and its derivatives, even if it is nonconvex. The `discretize' step uses a bounded finite element scheme to approximate the integral minimization problem with a convergent hierarchy of polynomial optimization problems over a compact feasible set, indexed by the decreasing size $h$ of the finite element mesh. The `relax' step employs sparse moment-sum-of-squares relaxations to approximate each polynomial optimization problem with a hierarchy of convex semidefinite programs, indexed by an increasing relaxation order $ω$. We prove that, as $ω\to\infty$ and $h\to 0$, solutions of such semidefinite programs provide approximate minimizers that converge in a suitable sense (including in certain $L^p$ norms) to the global minimizer of the original integral functional if it is unique. We also report computational experiments showing that our numerical strategy works well even when technical conditions required by our theoretical analysis are not satisfied.Open AccessGlobal minimization of polynomial integral functionals
We describe a `discretize-then-relax' strategy to globally minimize integral functionals over functions $u$ in a Sobolev space subject to Dirichlet boundary conditions. The strategy applies whenever the integral functional depends polynomially on $u$ and its derivatives, even if it is nonconvex. The `discretize' step uses a bounded finite element scheme to approximate the integral minimization problem with a convergent hierarchy of polynomial optimization problems over a compact feasible set, indexed by the decreasing size $h$ of the finite element mesh. The `relax' step employs sparse moment-sum-of-squares relaxations to approximate each polynomial optimization problem with a hierarchy of convex semidefinite programs, indexed by an increasing relaxation order $ω$. We prove that, as $ω\to\infty$ and $h\to 0$, solutions of such semidefinite programs provide approximate minimizers that converge in a suitable sense (including in certain $L^p$ norms) to the global minimizer of the original integral functional if it is unique. We also report computational experiments showing that our numerical strategy works well even when technical conditions required by our theoretical analysis are not satisfied. - Open Access
Finite element approximation of the Hardy constant
We consider finite element approximations to the optimal constant for the Hardy inequality with exponent $p=2$ in bounded domains of dimension $n=1$ or $n \geq 3$. For finite element spaces of piecewise linear and continuous functions on a mesh of size $h$, we prove that the approximate Hardy constant converges to the optimal Hardy constant at a rate proportional to $1/| \log h |^2$. This result holds in dimension $n=1$, in any dimension $n \geq 3$ if the domain is the unit ball and the finite element discretization exploits the rotational symmetry of the problem, and in dimension $n=3$ for general finite element discretizations of the unit ball. In the first two cases, our estimates show excellent quantitative agreement with values of the discrete Hardy constant obtained computationally.Open AccessFinite element approximation of the Hardy constant
We consider finite element approximations to the optimal constant for the Hardy inequality with exponent $p=2$ in bounded domains of dimension $n=1$ or $n \geq 3$. For finite element spaces of piecewise linear and continuous functions on a mesh of size $h$, we prove that the approximate Hardy constant converges to the optimal Hardy constant at a rate proportional to $1/| \log h |^2$. This result holds in dimension $n=1$, in any dimension $n \geq 3$ if the domain is the unit ball and the finite element discretization exploits the rotational symmetry of the problem, and in dimension $n=3$ for general finite element discretizations of the unit ball. In the first two cases, our estimates show excellent quantitative agreement with values of the discrete Hardy constant obtained computationally. - Open Access
- 1 January 2024
- Published by Banach Press LLC in Journal of Convex Analysis
- Vol. 31 (2) , 497-523
Finite Element Approximation of the Hardy Constant
We consider finite element approximations to the optimal constant for the Hardy inequality with exponent p=2 p = 2 in bounded domains of dimension n=1 n = 1 or n\geq 3 n ≥ 3 . For finite element spaces of piecewise linear and continuous functions on a mesh of size h h , we prove that the approximate Hardy constant converges to the optimal Hardy constant at a rate proportional to 1/|\log h|^2 1 / ∣ log h ∣ 2 . This result holds in dimension n=1 n = 1 , in any dimension n\geq 3 n ≥ 3 if the domain is the unit ball and the finite element discretization exploits the rotational symmetry of the problem, and in dimension n=3 n = 3 for general finite element discretizations of the unit ball. In the first two cases, our estimates show excellent quantitative agreement with values of the discrete Hardy constant obtained computationally.
- 1 January 2024
- Published by Banach Press LLC in Journal of Convex Analysis
- Vol. 31 (2) , 497-523
Open AccessFinite Element Approximation of the Hardy Constant
We consider finite element approximations to the optimal constant for the Hardy inequality with exponent p=2 p = 2 in bounded domains of dimension n=1 n = 1 or n\geq 3 n ≥ 3 . For finite element spaces of piecewise linear and continuous functions on a mesh of size h h , we prove that the approximate Hardy constant converges to the optimal Hardy constant at a rate proportional to 1/|\log h|^2 1 / ∣ log h ∣ 2 . This result holds in dimension n=1 n = 1 , in any dimension n\geq 3 n ≥ 3 if the domain is the unit ball and the finite element discretization exploits the rotational symmetry of the problem, and in dimension n=3 n = 3 for general finite element discretizations of the unit ball. In the first two cases, our estimates show excellent quantitative agreement with values of the discrete Hardy constant obtained computationally.