Thulasi Mylvaganam
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Thulasi Mylvaganam has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Mario Sassano
University of Rome Tor Vergata
14 shared publications - Alessandro Astolfi
King Abdullah University of Science and Technology (KAUST)
7 shared publications - Benita Nortmann
Department of Aeronautics, Imperial College London, London, U.K.
4 shared publications - Domenico Cappello
Department of Aeronautics, Imperial College London, London, U.K.
3 shared publications - Benita Nortmann
Department of Aeronautics Imperial College London SW7 2AZ London UK
2 shared publications Mario Sassano
2 shared publications
Latest Publications
- Open Access
Feedback Nash equilibria for scalar N-player linear quadratic dynamic games
Considering infinite-horizon, discrete-time, linear quadratic, N-player dynamic games with scalar dynamics, a graphical representation of feedback Nash equilibrium solutions is provided. This representation is utilised to derive conditions for the number and properties of different feedback Nash equilibria a game may admit. The results are illustrated via a numerical example.
Open AccessFeedback Nash equilibria for scalar N-player linear quadratic dynamic games
Considering infinite-horizon, discrete-time, linear quadratic, N-player dynamic games with scalar dynamics, a graphical representation of feedback Nash equilibrium solutions is provided. This representation is utilised to derive conditions for the number and properties of different feedback Nash equilibria a game may admit. The results are illustrated via a numerical example.
- Open Access
- 10 October 2024
- Published by Elsevier in Automatica
- Vol. 171
OL-NE for LQ differential games: A Port-Controlled Hamiltonian system perspective and some computational strategies
- 10 October 2024
- Published by Elsevier in Automatica
- Vol. 171
Open AccessOL-NE for LQ differential games: A Port-Controlled Hamiltonian system perspective and some computational strategies
- 6 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 8, 1451-1456
Feedback Nash Equilibrium Solutions of Two-Player LQ Differential Games: Synthesis and Analysis via a State/Costate Interpretation
Linear quadratic differential games and their feedback Nash equilibrium (F-NE) solutions are considered. First, it is shown that F-NE strategies can be derived from the restriction to an invariant subspace of a system that is reminiscent of the state/costate dynamics arising in the context of open-loop NE solutions. Second, in terms of synthesis, it is shown that the equilibrium subspace can be rendered externally stable via virtual inputs without modifying the underlying F-NE strategies. Building upon these findings, we propose a gradient descent algorithm to determine a solution of the coupled Algebraic Riccati Equations associated with F-NE, which are generally challenging to solve. Finally, in terms of analysis, we show that the F-NE strategy of each player can be interpreted as the output of a passive Port-Controlled Hamiltonian system, and that the behaviour of the original system under the action of the F-NE strategies can be interpreted as an interconnection of these.
- 6 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 8, 1451-1456
Feedback Nash Equilibrium Solutions of Two-Player LQ Differential Games: Synthesis and Analysis via a State/Costate Interpretation
Linear quadratic differential games and their feedback Nash equilibrium (F-NE) solutions are considered. First, it is shown that F-NE strategies can be derived from the restriction to an invariant subspace of a system that is reminiscent of the state/costate dynamics arising in the context of open-loop NE solutions. Second, in terms of synthesis, it is shown that the equilibrium subspace can be rendered externally stable via virtual inputs without modifying the underlying F-NE strategies. Building upon these findings, we propose a gradient descent algorithm to determine a solution of the coupled Algebraic Riccati Equations associated with F-NE, which are generally challenging to solve. Finally, in terms of analysis, we show that the F-NE strategy of each player can be interpreted as the output of a passive Port-Controlled Hamiltonian system, and that the behaviour of the original system under the action of the F-NE strategies can be interpreted as an interconnection of these.
- 13 May 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Control and Optimization
- Vol. 62 (3) , 1417-1436
Feedback and Open-Loop Nash Equilibria for LQ Infinite-Horizon Discrete-Time Dynamic Games
We consider dynamic games defined over an infinite horizon, characterized by linear, discrete-time dynamics and quadratic cost functionals. Considering such linear-quadratic dynamic games, we focus on their solutions in terms of Nash equilibrium strategies. Both feedback (F-NE) and open-loop (OL-NE) Nash equilibrium solutions are considered. The contributions of the paper are threefold. First, our detailed study reveals some interesting structural insights in relation to F-NE solutions. Second, as a stepping stone toward our consideration of OL-NE strategies, we consider a specific infinite-horizon discrete-time (single-player) optimal control problem, wherein the dynamics are influenced by a known exogenous input and draw connections between its solution obtained via dynamic programming and Pontryagin’s minimum principle. Finally, we exploit the latter result to provide a characterization of OL-NE strategies of the class of infinite-horizon dynamic games. The results and key observations made throughout the paper are illustrated via a numerical example.
- 13 May 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Control and Optimization
- Vol. 62 (3) , 1417-1436
Feedback and Open-Loop Nash Equilibria for LQ Infinite-Horizon Discrete-Time Dynamic Games
We consider dynamic games defined over an infinite horizon, characterized by linear, discrete-time dynamics and quadratic cost functionals. Considering such linear-quadratic dynamic games, we focus on their solutions in terms of Nash equilibrium strategies. Both feedback (F-NE) and open-loop (OL-NE) Nash equilibrium solutions are considered. The contributions of the paper are threefold. First, our detailed study reveals some interesting structural insights in relation to F-NE solutions. Second, as a stepping stone toward our consideration of OL-NE strategies, we consider a specific infinite-horizon discrete-time (single-player) optimal control problem, wherein the dynamics are influenced by a known exogenous input and draw connections between its solution obtained via dynamic programming and Pontryagin’s minimum principle. Finally, we exploit the latter result to provide a characterization of OL-NE strategies of the class of infinite-horizon dynamic games. The results and key observations made throughout the paper are illustrated via a numerical example.
- Open Access
- 8 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 69 (10) , 6561-6575
Nash Equilibria for Linear Quadratic Discrete-Time Dynamic Games via Iterative and Data-Driven Algorithms
Determining feedback Nash equilibrium solutions of nonzero-sum dynamic games is generally challenging. In this paper, we propose four different iterative algorithms to find Nash equilibrium strategies for discrete-time linear quadratic games. The strategy update laws are based on the solution of either Lyapunov or Riccati equations for each player. Local convergence criteria are discussed. Motivated by the fact that in many practical scenarios each player in the game may have access to different (incomplete) information, we also introduce purely data-driven implementations of the algorithms. This allows the players to reach a Nash equilibrium solution of the game via scheduled experiments and without knowledge of each other's performance criteria or of the system dynamics. The efficacy of the presented algorithms is illustrated via numerical examples and a practical example involving human-robot interaction.
- 8 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 69 (10) , 6561-6575
Open AccessNash Equilibria for Linear Quadratic Discrete-Time Dynamic Games via Iterative and Data-Driven Algorithms
Determining feedback Nash equilibrium solutions of nonzero-sum dynamic games is generally challenging. In this paper, we propose four different iterative algorithms to find Nash equilibrium strategies for discrete-time linear quadratic games. The strategy update laws are based on the solution of either Lyapunov or Riccati equations for each player. Local convergence criteria are discussed. Motivated by the fact that in many practical scenarios each player in the game may have access to different (incomplete) information, we also introduce purely data-driven implementations of the algorithms. This allows the players to reach a Nash equilibrium solution of the game via scheduled experiments and without knowledge of each other's performance criteria or of the system dynamics. The efficacy of the presented algorithms is illustrated via numerical examples and a practical example involving human-robot interaction.
- Open Access
Feedback and Open-Loop Nash Equilibria for LQ Infinite-Horizon Discrete-Time Dynamic Games
We consider dynamic games defined over an infinite horizon, characterized by linear, discrete-time dynamics and quadratic cost functionals. Considering such linear-quadratic (LQ) dynamic games, we focus on their solutions in terms Nash equilibrium strategies. Both Feedback (F-NE) and Open-Loop (OL-NE) Nash equilibrium solutions are considered. The contributions of the paper are threefold. First, our detailed study reveals some interesting structural insights in relation to F-NE solutions. Second, as a stepping stone towards our consideration of OL-NE strategies, we consider a specific infinite-horizon discrete-time (single-player) optimal control problem, wherein the dynamics are influenced by a known exogenous input and draw connections between its solution obtained via Dynamic Programming and Pontryagin's Minimum Principle. Finally, we exploit the latter result to provide a characterization of OL-NE strategies of the class of infinite-horizon dynamic games. The results and key observations made throughout the paper are illustrated via a numerical example.
Open AccessFeedback and Open-Loop Nash Equilibria for LQ Infinite-Horizon Discrete-Time Dynamic Games
We consider dynamic games defined over an infinite horizon, characterized by linear, discrete-time dynamics and quadratic cost functionals. Considering such linear-quadratic (LQ) dynamic games, we focus on their solutions in terms Nash equilibrium strategies. Both Feedback (F-NE) and Open-Loop (OL-NE) Nash equilibrium solutions are considered. The contributions of the paper are threefold. First, our detailed study reveals some interesting structural insights in relation to F-NE solutions. Second, as a stepping stone towards our consideration of OL-NE strategies, we consider a specific infinite-horizon discrete-time (single-player) optimal control problem, wherein the dynamics are influenced by a known exogenous input and draw connections between its solution obtained via Dynamic Programming and Pontryagin's Minimum Principle. Finally, we exploit the latter result to provide a characterization of OL-NE strategies of the class of infinite-horizon dynamic games. The results and key observations made throughout the paper are illustrated via a numerical example.
- Open Access
Direct data-driven control of LTV systems
Considering discrete-time linear time-varying systems with unknown dynamics, controllers guaranteeing bounded closed-loop trajectories, optimal performance and robustness to process and measurement noise are designed via convex feasibility and optimisation problems involving purely data-dependent linear matrix inequalities. For the special case of periodically time-varying systems, infinite-horizon guarantees are achieved based on finite-length data sequences.
Open AccessDirect data-driven control of LTV systems
Considering discrete-time linear time-varying systems with unknown dynamics, controllers guaranteeing bounded closed-loop trajectories, optimal performance and robustness to process and measurement noise are designed via convex feasibility and optimisation problems involving purely data-dependent linear matrix inequalities. For the special case of periodically time-varying systems, infinite-horizon guarantees are achieved based on finite-length data sequences.
- 16 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 68 (8) , 4888-4895
Direct Data-Driven Control of Linear Time-Varying Systems
Considering discrete-time linear time-varying systems with unknown dynamics, controllers guaranteeing bounded closed-loop trajectories, optimal performance and robustness to process and measurement noise are designed via convex feasibility and optimisation problems involving purely data-dependent linear matrix inequalities. For the special case of periodically time-varying systems, infinite-horizon guarantees are achieved based on finite-length data sequences.
- 16 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 68 (8) , 4888-4895
Direct Data-Driven Control of Linear Time-Varying Systems
Considering discrete-time linear time-varying systems with unknown dynamics, controllers guaranteeing bounded closed-loop trajectories, optimal performance and robustness to process and measurement noise are designed via convex feasibility and optimisation problems involving purely data-dependent linear matrix inequalities. For the special case of periodically time-varying systems, infinite-horizon guarantees are achieved based on finite-length data sequences.
- Open Access
- 20 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 68 (10) , 5954-5965
Finite-Dimensional Characterization of Optimal Control Laws Over an Infinite Horizon for Nonlinear Systems
Infinite-horizon optimal control problems for nonlinear systems are considered. Due to the nonlinear and intrinsically infinite-dimensional nature of the task, solving such optimal control problems is challenging. In this paper an exact finite-dimensional characterisation of the optimal solution over the entire horizon is proposed. This is obtained via the (static) minimisation of a suitably defined function of (projected) trajectories of the underlying Hamiltonian dynamics on a hypersphere of fixed radius. The result is achieved in the spirit of the so-called shooting methods by introducing, via simultaneous forward/backward propagation, an intermediate shooting point much closer to the origin, regardless of the actual initial state. A modified strategy allows one to determine an arbitrarily accurate approximate solution by means of standard gradient-descent algorithms over compact domains. Finally, to further increase robustness of the control law, a receding-horizon architecture is envisioned by designing a sequence of shrinking hyperspheres. These aspects are illustrated by means of a benchmark numerical simulation.
- 20 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 68 (10) , 5954-5965
Open AccessFinite-Dimensional Characterization of Optimal Control Laws Over an Infinite Horizon for Nonlinear Systems
Infinite-horizon optimal control problems for nonlinear systems are considered. Due to the nonlinear and intrinsically infinite-dimensional nature of the task, solving such optimal control problems is challenging. In this paper an exact finite-dimensional characterisation of the optimal solution over the entire horizon is proposed. This is obtained via the (static) minimisation of a suitably defined function of (projected) trajectories of the underlying Hamiltonian dynamics on a hypersphere of fixed radius. The result is achieved in the spirit of the so-called shooting methods by introducing, via simultaneous forward/backward propagation, an intermediate shooting point much closer to the origin, regardless of the actual initial state. A modified strategy allows one to determine an arbitrarily accurate approximate solution by means of standard gradient-descent algorithms over compact domains. Finally, to further increase robustness of the control law, a receding-horizon architecture is envisioned by designing a sequence of shrinking hyperspheres. These aspects are illustrated by means of a benchmark numerical simulation.
- Open Access
- 17 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 68 (5) , 2683-2698
Model-Based Policy Iterations for Nonlinear Systems via Controlled Hamiltonian Dynamics
The infinite-horizon optimal control problem for nonlinear systems is studied. In the context of model-based, iterative learning strategies we propose an alternative definition and construction of the temporal difference error arising in Policy Iteration strategies. In such architectures the error is computed via the evolution of the Hamiltonian function (or, possibly, of its integral) along the trajectories of the closed-loop system. Herein the temporal difference error is instead obtained via two subsequent steps: first the dynamics of the underlying costate variable in the Hamiltonian system is steered by means of a (virtual) control input in such a way that the stable invariant manifold becomes externally attractive. Then, the distance-from-invariance of the manifold, induced by approximate solutions, yields a natural candidate measure for the policy evaluation step. The policy improvement phase is then performed by means of standard gradient descent methods that allows to correctly update the weights of the underlying functional approximator. The above architecture then yields an iterative (episodic) learning scheme based on a scalar, constant reward at each iteration, the value of which is insensitive to the length of the episode, as in the original spirit of Reinforcement Learning strategies for discrete-time systems. Finally, the theory is validated by means of a numerical simulation involving an automatic flight control problem.
- 17 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 68 (5) , 2683-2698
Open AccessModel-Based Policy Iterations for Nonlinear Systems via Controlled Hamiltonian Dynamics
The infinite-horizon optimal control problem for nonlinear systems is studied. In the context of model-based, iterative learning strategies we propose an alternative definition and construction of the temporal difference error arising in Policy Iteration strategies. In such architectures the error is computed via the evolution of the Hamiltonian function (or, possibly, of its integral) along the trajectories of the closed-loop system. Herein the temporal difference error is instead obtained via two subsequent steps: first the dynamics of the underlying costate variable in the Hamiltonian system is steered by means of a (virtual) control input in such a way that the stable invariant manifold becomes externally attractive. Then, the distance-from-invariance of the manifold, induced by approximate solutions, yields a natural candidate measure for the policy evaluation step. The policy improvement phase is then performed by means of standard gradient descent methods that allows to correctly update the weights of the underlying functional approximator. The above architecture then yields an iterative (episodic) learning scheme based on a scalar, constant reward at each iteration, the value of which is insensitive to the length of the episode, as in the original spirit of Reinforcement Learning strategies for discrete-time systems. Finally, the theory is validated by means of a numerical simulation involving an automatic flight control problem.