Lionel Mathelin

Neural ordinary differential equations offer an effective framework for modeling dynamical systems by learning a continuous-time vector field. However, they rely on the Markovian assumption - that future states depend only on the current state - which is often untrue in real-world scenarios where the dynamics may depend on the history of past states. This limitation becomes especially evident in settings involving the continuous control of complex systems with delays and memory effects. To capture historical dependencies, existing approaches often rely on recurrent neural network (RNN)-based encoders, which are inherently discrete and struggle with continuous modeling. In addition, they may exhibit poor training behavior. In this work, we investigate the use of the signature transform as an encoder for learning non-Markovian dynamics in a continuous-time setting. The signature transform offers a continuous-time alternative with strong theoretical foundations and proven efficiency in summarizing multidimensional information in time. We integrate a signature-based encoding scheme into encoder-decoder dynamics models and demonstrate that it outperforms RNN-based alternatives in test performance on synthetic benchmarks.

Flow control has attracted research for the positive outcomes in engineering systems applications and is finding renewed interest thanks to the rise of data-driven modeling and control algorithms. Interestingly, due to the nonlinearities and the strong convective nature leading to time delays, fluid systems can also serve as challenging test-bed for the further development of control algorithms; indeed, despite the huge potential, the limitations introduced by the plant represent a challenge and the inherent complexity of the dynamics at play requires appropriate strategies during the training process. In this contribution, we consider a reinforcement learning framework and introduce a well-suited actor-critic algorithm to tackle these challenges. The presented algorithm is i) data-parsimonious, ii) leverages optimistic order in the value iteration and iii) is equipped with policy improvement-based error bounds. These features allow a speed-up of the convergence and provide a theoretical-based stopping criterion. As test cases, we consider a linearized version of the Kuramoto-Sivashinsky equation and the control of instabilities in a two-dimensional boundary-layer flow developing over a flat plate, by introducing a minimal number of sensors and actuators. Concerning analogous works that appeared in literature, we show that keeping a rather classical plant is sufficient for guaranteeing adequate performance if the input-output dynamics as well as the observability properties are taken into account.

Many successful methods to learn dynamical systems from data have recently been introduced. Such methods often rely on the availability of the system's full state. However, this underlying hypothesis is rather restrictive as it is typically not confirmed in practice, leaving us with partially observed systems. Utilizing the Mori-Zwanzig (MZ) formalism from statistical physics, we demonstrate that Constant Lag Neural Delay Differential Equations (ND-DEs) naturally serve as suitable models for partially observed states. In empirical evaluation, we show that such models outperform existing methods on both synthetic and experimental data.