Zuhua Xu

At an urban intersection, traffic performance is highly influenced not only by signal timing schemes but also by lane assignments. However, dynamic adjustment of lane configurations remains understudied due to the prevalent use of predetermined lane arrangements. To address this research gap, we propose the TransLane framework, which collaboratively optimizes time and space resources within an intersection by implementing dynamic lanes. We model each intersection as an agent and apply a hierarchical recommendation approach to these agents. Additionally, the framework addresses the challenge of training an agent in the absence of sufficient data by using transfer learning techniques. Specifically, a new intersection agent can be trained from pre-trained agents via model and sample transfer mechanisms, which facilitate the reuse of traffic control knowledge from similar intersections. The efficacy and superiority of TransLane are demonstrated through simulation studies that show it outperforms models that optimize space or time resources independently using widely adopted traffic control optimization methods.

This paper investigates an adaptive optimal admittance control scheme for robot manipulators interacting with unknown environment. To resolve the optimized interaction performance considering tracking error and interaction force, an impedance adaptation approach is developed without the initial stabilizing policy. Based on the gradient‐based updating method, the online solution can exponentially converge to the optimal impedance gain without prior knowledge of environment dynamics. A nonlinear mapping method is integrated into the admittance control, transforming the constrained system into an equivalent system without state constraints. By eliminating feasibility conditions, the tracking controller can achieve the full‐state asymmetric time‐varying constraints under a broad range of initial conditions. Through the Lyapunov analysis, it is proven that the closed‐loop signals are bounded. Finally, simulation and experiment results demonstrate the effectiveness of the proposed methods.

At present, working conditions are becoming more complex, and operation policy requirements are more diverse in process system engineering. To control a process problem, a balance must be found between speed and stability, in that operations should sometimes be faster and other times smoother. Traditional controllers, such as PID and model predictive control are applied in various problems, and some parameters in controllers can be used to represent the operation policy. However, there can be difficulties in tuning parameters, and time costs of online calculation. This article proposes parametric deep reinforcement learning (PDRL) to replace traditional controllers. PDRL has two parts. A vanilla DRL framework is adapted to solve the setpoint tracking problem. With a state and a reward function and robust training tricks, trained agents can be applied to more generalized working conditions. Base agents of different operation policies are trained in advance. With target performance from operators, the target policy can be fitted by base agents with a set of weights, which are first optimized by minimizing the squared error between the target and fitted policy in a basic task, and applied to generalized conditions. A shell benchmark problem is chosen as a case study, whose results show that PDRL has feasibility and stability both in basic and generalized tasks, even in a noisy environment.