Moosa Ayati

1. Many real-world systems experience input delay due to physical plant characteristics, transportation lag, computational latency, or communication constraints. Delays can significantly degrade sy...

This paper presents an integrated hybrid reinforcement learning–model predictive control (RLMPC) framework for autonomous highway systems, unifying macroscopic traffic flow regulation and microscopic on‐ramp merging control. At the macroscopic level, a ramp metering (RM) controller based on a data‐driven model predictive control (MPC) formulation using second‐order Q‐learning is implemented in the METANET environment on a benchmark three‐segment freeway without the need for explicit traffic models. The RLMPC RM learns optimal flow regulation directly from closed‐loop data, achieving enhanced system performance, constraint satisfaction and smooth control compared to common RM algorithms such as ALINEA, MPC and deep RL. At the microscopic level, an RLMPC merging controller manages autonomous on‐ramp manoeuvres in which an ego vehicle enters the mainline approximately 160 m before the merge point and completes the manoeuvre 50 m downstream while interacting with surrounding vehicles. In this phase, when a collision risk arises, the MPC takes control; otherwise, the reinforcement learning (RL) policy operates, combining model‐based safety with learning‐based efficiency and yielding superior overall performance. Evaluations under varied traffic conditions show that implementing RM at the macroscopic level significantly improves microscopic on‐ramp merging performance. Relative to the no‐RM baseline, the framework achieves a 34.5% reduction in merge time under slow traffic conditions, eliminates collision events and moderately enhances overall efficiency and driving comfort.

This study proposes a new adaptive integral terminal sliding mode control (AITSMC) with input saturation for a vehicle steer‐by‐wire (SbW) system. A fast disturbance observer compensates for uncertainties, external disturbances, and input saturation nonlinearities. An adaptation mechanism is applied to improve the performance of the controller which makes the designed control scheme more applicable to control an SbW system. Based on the Lyapunov stability theorem, the finite‐time convergence of the disturbance observer and the output tracking error is proved in the presence of disturbance and nonsymmetric actuator saturation. At last, using the co‐simulation platform of CarSim® and MATLAB/Simulink, two numerical simulations are carried out under challenging maneuvers and road conditions to examine the controller's performance. The results confirm its sufficiency regarding accuracy and robustness for the SbW system.

This paper introduces a novel framework for anxiety‐responsive autonomous driving by integrating online real‐time anxiety estimation within a simulation‐based proof‐of‐concept with deep reinforcement learning‐based control. First, we develop an anxiety estimation model using a random forest classifier trained on vehicle kinematic features—speed, acceleration, and inter‐vehicle distance—from a publicly available dataset, achieving a classification accuracy of 0.86 on self‐reported anxiety levels. By relying only on kinematic information, the proposed model enables anxiety estimation without requiring physiological sensors during simulation‐based control. Next, this estimator is embedded within a Deep Q‐Network (DQN) controller by augmenting the reward function with an anxiety‐based penalty term. Simulations conducted in the HighwayEnv environment demonstrate that the resulting anxiety‐aware agent learns a measurably more defensive driving style—characterised by lower speeds and smoother manoeuvres—compared to a conventional DQN baseline evaluated under the same simulation setting. This provides an explicit mechanism for incorporating passenger anxiety into autonomous driving control and highlights the novelty of linking anxiety estimation directly to reward‐guided motion planning. By demonstrating that an explicit model of occupant anxiety can directly shape an AV's motion planning policy in simulation, this work provides a foundational framework for developing more human‐centric and trustworthy autonomous systems.

In this article, a novel mathematical model of a quadrotor UAV (Unmanned Aerial Vehicle) suffering from one type of structural fault, angle deflection in a motor and the corresponding mechanical link, has been derived. Such faults add additional nonlinear terms to the differential equations of motion and change the 3D configuration of the UAV. These nonlinear terms rely significantly on the unknown fault angles, which are estimated via a Radial Basis Function Neural Network (RBFNN). Moreover, a Non‐singular Fast Terminal Sliding Mode Control (NFTSMC) scheme optimized by a Genetic Algorithm (GANFTSMC) has been designed for trajectory tracking and reduction of control effort. Simulation results for the entire closed‐loop system, using three different types of reference signals and fault angles, demonstrate the significant performance of the proposed controller in the presence of structural faults and external disturbances. Furthermore, the settling time of error dynamics of the system states to the origin has been finite, and the superior performance of our proposed control strategy has been validated via comparison with other robust nonlinear control techniques implemented in the literature.

Due to the high nonlinearity of the quadrotor dynamics and the generated structural faults, a Fault‐Observer‐Based Fault‐Tolerant Fuzzy Terminal Sliding Mode Controller (FT2SMC) is utilized. This strategic utilization remains effective even when the system encounters input saturation. A fault observer estimates the fault vector since it is applied to the governing equations of motion as external disturbances. Additionally, a Fuzzy Inference System (FIS) has been employed to estimate the discontinuous term of the terminal sliding mode control and to alleviate the chattering phenomenon. The finite‐time convergence of the tracking errors of the attitude and the altitude toward the origin has been guaranteed, and the entire closed‐loop stability of the faulty quadrotor has been proved using Lyapunov theory, ensuring that it continues following the desired trajectory even in the presence of faults. The simulation results indicate the prodigious effectiveness of the controller, which navigates the quadrotor to a desired altitude. Also, the superiority of this control algorithm is demonstrated by comparing the results of the control inputs with those obtained with a regular terminal sliding mode controller.

This paper presents a novel approach utilizing a sliding mode controller to control a quadrotor effectively in the presence of structural faults. While previous research has extensively explored fault-tolerant control for UAVs, particularly addressing faulty actuators and sensors, there remains a significant gap in addressing control methods for quadrotors experiencing structural faults. We focus specifically on scenarios where a fault in one of the quadrotor’s rotors diverts thrust from the vertical axis, complicating traditional control methods. Our proposed method involves two key steps: firstly, identifying the fault vector using a terminal sliding mode observer, and secondly, calculating appropriate control inputs using a sliding mode method based on the estimated fault vector. Simulation results demonstrate the robustness of this approach, showing accurate control performance even in the presence of multiple faults and disturbances. The system’s reliability hinges on the convergence of the disturbance observer’s estimation error to zero within a finite time frame. In conclusion, the developed controller offers a dependable solution for safely controlling a faulty quadrotor amidst high nonlinearity and uncertainty.

In this paper, a predictive deep reinforcement learning-based control algorithm is introduced for the cooperative adaptive cruise control (CACC) system. The proposed control algorithm achieves an optimal response in the interaction of the host vehicle with the leading vehicles. For this purpose, at first, a new algorithm based on deep neural networks is proposed for modeling and predicting the driving behavior of the leading vehicle, according to the traffic behavior of its surrounding vehicles. Then, the outputs of the predictive model are used as additional states for the upper-level controller of the CACC system, which is designed based on a deep reinforcement learning approach. Therefore, the proposed controller uses conventional states such as relative distance and speed of the host vehicle along with the leading vehicles and employs the speed profile of the leading vehicle in the future time horizons to train the reinforcement learning agent. Thus, the upper-level controller learning process is improved. In addition, for regulating the lower-level controller of CACC system against parametric uncertainties in electric vehicle dynamics, a deep reinforcement learning-based controller is proposed, which can be retuned according to the real-time estimated parameters of vehicle mass and road grade. Simulation results represent the promising performance of the proposed control algorithm for maintaining a balance between travel safety, passenger comfort, and reducing vehicle energy consumption.