Shaoshuai Mou

This paper addresses the problem of solving systems of nonlinear equations in large-scale networked settings. While distributed methods for linear equations are well established, extending these approaches to nonlinear equations remains challenging due to dimensionality constraints, global information requirements, and the reliance of existing clustered architectures on hierarchical aggregator nodes. We develop a distributed algorithm for nonlinear equation solving based on a single-layer clustered multi-agent systems (MAS) that eliminates the need for central aggregators. The nonlinear function under consideration is assumed to decompose additively across clusters, with each cluster-level function consisting of local components available only to individual agents. The proposed framework integrates intra-cluster consensus with inter-cluster conservation to achieve cooperative convergence. Numerical simulations are provided to validate the effectiveness of the proposed distributed algorithm.

Switched systems pose a challenge for conventional deterministic dynamic optimization methods given the requirement for an accurate process model. In this paper, we propose a model-free reinforcement learning algorithm for optimal control of switched systems with provable local convergence. First, we develop a parameterized-action Markov decision process formulation that rigorously follows the definition of switched systems with internally forced switching. Second, we consider the parameterized optimization problem where the continuous input is the parameter and the optimization is over the set of feasible switching modes. From this point of view, we give the sufficient conditions under which the optimal value function is differentiable and the formula for its derivative. Third, we propose a bi-level algorithm for locating the optimal input and switching law, by alternately employing Q-learning and policy gradient methods guided by the optimal value function. The local convergence property of the algorithm is mathematically proved. Finally, we illustrate the advantages of our problem formulation and the effectiveness of our algorithm through case studies based on practical applications. Note to Practitioners—Many practical systems switch between multiple operating modes. Optimal control of such systems is challenging because the problem consists of both continuous and discrete decisions, and conventional model-based methods require accurate dynamics for each mode. This paper presents a model-free reinforcement learning approach for optimal control of switched systems. The method integrates Q-learning with policy gradient method in a bi-level framework. By learning directly via the interaction with the environment, the approach can locate the optimal control input and switching sequences without prior knowledge of the system model. The approach is demonstrated on three case studies based on real engineering scenarios. In these examples, the learned policy improved performance without prior-known dynamic models. For practitioners, the key benefit is that this approach applies to systems with switching mechanisms and does not require precise modeling, making it suitable for complex optimal control tasks where accurate modeling is costly or impractical. However, the approach requires a safe environment for online training, which implies future research directions on offline reinforcement learning using collected data, and transfer learning to mitigate model mismatch between simulation and real-world deployment.

A correction is given to a previously published result concerned with the relationship between a suitably defined matrix seminorm for consensus analysis and a coefficient of ergodicity.

This paper proposes a Safe Online Control-Informed Learning framework for safety-critical autonomous systems. The framework unifies optimal control, parameter estimation, and safety constraints into an online learning process. It employs an extended Kalman filter to incrementally update system parameters in real time, enabling robust and data-efficient adaptation under uncertainty. A softplus barrier function enforces constraint satisfaction during learning and control while eliminating the dependence on high-quality initial guesses. Theoretical analysis establishes convergence and safety guarantees, and the framework’s effectiveness is demonstrated on cart-pole and robot-arm systems.

The design of the performance index, also referred to as cost or reward shaping, is central to both optimal control and reinforcement learning, as it directly determines the behaviors, trade-offs, and objectives that the resulting control laws seek to achieve. A commonly used approach for this inference task in recent years is differentiable trajectory optimization, which allows gradients to be computed with respect to cost parameters by differentiating through an optimal control solver. However, this method often requires repeated solving of the underlying optimal control problem at every iteration, making the method computationally expensive. In this work, assuming known dynamics, we propose a novel framework that analytically links the performance index to the resulting closed-loop optimal control law, thereby transforming a typically bi-level inverse problem into a tractable single-level formulation. Our approach is motivated by the question: given a closed-loop control law that solves an infinite-horizon optimal control problem, how does this law change when the performance index is modified with additional terms? This formulation yields closed-form characterizations for broad classes of systems and performance indices, which not only facilitate interpretation and stability analysis, but also provide insight into the robust stability and input-to-state stable behavior of the resulting nonlinear closed-loop system. Moreover, this analytical perspective enables the generalization of our approach to diverse design objectives, yielding a unifying framework for performance index shaping. Given specific design objectives, we propose a systematic methodology to guide the shaping of the performance index and thereby design the resulting optimal control law.

Electric vertical take-off and landing (eVTOL) aircraft have emerged as a promising solution to transform urban transportation. They present a few technical challenges for battery management, a prominent one of which is the prediction of the power capability of their lithium-ion battery systems. The challenge originates from the high C-rate discharging conditions required during eVTOL flights as well as the complexity of lithium-ion batteries' electro-thermal dynamics. This paper, for the first time, formulates a power limit prediction problem for eVTOL which explicitly considers long prediction horizons and the possible occurrence of emergency landings. We then harness machine learning to solve this problem in two intertwined ways. First, we adopt a dynamic model that integrates physics with machine learning to predict a lithium-ion battery's voltage and temperature behaviors with high accuracy. Second, while performing search for the maximum power, we leverage machine learning to predict the remaining discharge time and use the prediction to accelerate the search with fast computation. Our validation results show the effectiveness of the proposed study for eVTOL operations.