Liangfa Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Liangfa Chen has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
- Jingliang Duan
School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
3 shared publications - Liming Xiao
School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
3 shared publications - Shengbo Eben Li
Tsinghua University
3 shared publications - Wenxuan Wang
School of Vehicle and Mobility, Tsinghua University, Beijing, China
2 shared publications - Bo Cheng
School of Vehicle and Mobility, Tsinghua University, Beijing, China
1 shared publication - Fawang Zhang
School of Mechanical Engineering, Beijing Institute of Technology, Beijing, China
1 shared publication
Latest Publications
- 19 March 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 319
A smooth reinforcement learning method for trajectory tracking and collision avoidance of wheeled vehicle
- 19 March 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 319
A smooth reinforcement learning method for trajectory tracking and collision avoidance of wheeled vehicle
- 21 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. PP (23798858) , 1-10
Smooth Filtering Neural Network for Reinforcement Learning
Reinforcement learning (RL) has demonstrated considerable potential in addressing intricate control and decision problems such as vehicle tracking control and obstacle avoidance. Nonetheless, the control policies acquired through RL often lack smoothness even in the presence of minor noises or disturbances, which may induce oscillations, overheating, and even damage to the system in real-world applications. Existing methods handle this issue from temporal or spatial domains and are often tightly coupled with specific tasks. This paper studies the smoothness of control policy from the frequency domain perspective. We propose a class of neural networks with low-pass filtering ability, named Smonet, to alleviate the non-smooth issue by learning a low-frequency representation within hidden layers. Smonet features with serial filtering layers responsible for low-pass filtering of the input signal. Each filtering layer contains multiple inertia cells, one adaptive cell, and one activation layer. To facilitate the filtering ability of Smonet, we further proposed a Smonet-based RL training method by integrating an extra regularization term relating to filtering factors to standard RL loss. Finally, we assess the efficacy of Smonet through diverse simulated robot control tasks and a real-world mobile robot obstacle avoidance experiment, comparing its performance with two commonly utilized networks, multi-layer perceptron and gated recurrent unit. Results indicate that Smonet consistently enhances policy smoothness under various observation noises without compromising control performance. Notably, it achieves up to a 72.7% reduction in the action fluctuation ratio compared to traditional network structures.
- 21 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. PP (23798858) , 1-10
Smooth Filtering Neural Network for Reinforcement Learning
Reinforcement learning (RL) has demonstrated considerable potential in addressing intricate control and decision problems such as vehicle tracking control and obstacle avoidance. Nonetheless, the control policies acquired through RL often lack smoothness even in the presence of minor noises or disturbances, which may induce oscillations, overheating, and even damage to the system in real-world applications. Existing methods handle this issue from temporal or spatial domains and are often tightly coupled with specific tasks. This paper studies the smoothness of control policy from the frequency domain perspective. We propose a class of neural networks with low-pass filtering ability, named Smonet, to alleviate the non-smooth issue by learning a low-frequency representation within hidden layers. Smonet features with serial filtering layers responsible for low-pass filtering of the input signal. Each filtering layer contains multiple inertia cells, one adaptive cell, and one activation layer. To facilitate the filtering ability of Smonet, we further proposed a Smonet-based RL training method by integrating an extra regularization term relating to filtering factors to standard RL loss. Finally, we assess the efficacy of Smonet through diverse simulated robot control tasks and a real-world mobile robot obstacle avoidance experiment, comparing its performance with two commonly utilized networks, multi-layer perceptron and gated recurrent unit. Results indicate that Smonet consistently enhances policy smoothness under various observation noises without compromising control performance. Notably, it achieves up to a 72.7% reduction in the action fluctuation ratio compared to traditional network structures.
- 24 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 10 (3) , 1759-1770
Multi-Style Distributional Soft Actor-Critic: Learning a Unified Policy for Diverse Control Behaviors
Reinforcement learning (RL) has excelled in sequential decision-making and control tasks, yet traditional RL algorithms are limited by adherence to a single control style in identical scenarios, failing to address varied control preferences. Existing multi-style RL methods typically require customized reward or objective functions tailored to specific control styles, which may not be feasible when diverse driving styles are necessary. To overcome these limitations, we propose the multi-style distributional soft actor-critic (M-DSAC) algorithm, capable of learning a single policy that supports multiple control behaviors. We begin by developing a multi-style policy iteration (MPI) framework that learns the entire distribution of returns, known as the value distribution, rather than just focusing on the expected return (i.e., the $Q$ value). In this framework, we utilize the quantile index of the value distribution as a style indicator, enhancing the inputs of both the policy and its corresponding value distribution with these quantile indices. Building upon the MPI framework, the M-DSAC algorithm employs a parameterized diagonal Gaussian function to approximate the value distribution. This approach enables efficient computation of different value quantiles by combining the value distribution's mean and standard deviations with appropriate coefficients. By optimizing the policy across different quantiles, M-DSAC efficiently learns a versatile policy that can handle a range of control styles without the burden of significant computing costs. Experimental evaluations using MuJoCo benchmarks and real-world robot control tasks confirm the effectiveness of M-DSAC, showcasing its broad practical applicability.
- 24 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 10 (3) , 1759-1770
Multi-Style Distributional Soft Actor-Critic: Learning a Unified Policy for Diverse Control Behaviors
Reinforcement learning (RL) has excelled in sequential decision-making and control tasks, yet traditional RL algorithms are limited by adherence to a single control style in identical scenarios, failing to address varied control preferences. Existing multi-style RL methods typically require customized reward or objective functions tailored to specific control styles, which may not be feasible when diverse driving styles are necessary. To overcome these limitations, we propose the multi-style distributional soft actor-critic (M-DSAC) algorithm, capable of learning a single policy that supports multiple control behaviors. We begin by developing a multi-style policy iteration (MPI) framework that learns the entire distribution of returns, known as the value distribution, rather than just focusing on the expected return (i.e., the $Q$ value). In this framework, we utilize the quantile index of the value distribution as a style indicator, enhancing the inputs of both the policy and its corresponding value distribution with these quantile indices. Building upon the MPI framework, the M-DSAC algorithm employs a parameterized diagonal Gaussian function to approximate the value distribution. This approach enables efficient computation of different value quantiles by combining the value distribution's mean and standard deviations with appropriate coefficients. By optimizing the policy across different quantiles, M-DSAC efficiently learns a versatile policy that can handle a range of control styles without the burden of significant computing costs. Experimental evaluations using MuJoCo benchmarks and real-world robot control tasks confirm the effectiveness of M-DSAC, showcasing its broad practical applicability.