Yee-Ming Ooi

Reinforcement Learning (RL) enables learning optimal decision-making strategies by maximizing cumulative rewards. Deep reinforcement learning (DRL) enhances this process by integrating deep neural networks (DNNs) for effective feature extraction from high-dimensional input data. Unlike prior studies focusing on algorithm design, we investigated the impact of different feature extractors, DNNs, on DRL performance. We propose an enhanced feature extraction model to improve control effectiveness based on the proximal policy optimization (PPO) framework in autonomous driving scenarios. Through a comparative analysis of well-known convolutional neural networks (CNNs), MobileNet, SqueezeNet, and ResNet, the experimental results demonstrate that our model achieves higher cumulative rewards and better control stability, providing valuable insights for DRL applications in autonomous systems.

Given that the Global Positioning System (GPS) may not always be precise or available, it would be advantageous for drone positioning to utilize a vision-based approach. It analyzes geometric features to determine the drone’s position and identify its flight path. In the literature, an existing vision-based study is proposed to position the drone by a shallow Convolutional Neural Network (CNN). However, they only consider a constant viewpoint (towards the north). This is not practical when implemented in the real world. Hence, we start to examine the impact of viewpoint variations. While the drone’s cameras capture images, these changing viewpoints will affect the accuracy and stability of models. In this paper, MobileNetV2 is the base for creating a classification mechanism for accurate and stable position estimation. Our findings illustrate the importance and relationship of various training data among the different performances of the model. The model achieves only 17% accuracy without applying rotational augmentation in the most practical scenario. However, the accuracy significantly improves by incorporating rotational augmentation with 15-degree increments to 95%.

A positioning system in a specific space is for the purpose of determining the location of an object. The Global Positioning System (GPS) is the most popular and valuable development for classical navigation. However, it may not always be precise and available due to the effects of multi-path propagation and signal attenuation. Thus, we need some additional skills to keep its stability and accuracy or improve its performance. Based on the technique of neural networks, the positioning information from multiple GPS receivers is composed to obtain a better version which is more accurate and stable and thus can be applied to advanced applications. Particularly, the concepts of shallow and asymmetric neural networks are used in this work. Our design possesses fewer hidden layers via the former property and further reduces the connections of classical Fully Connected Neural Networks (FCNNs) via the latter property. Hence, it takes very little time to realize the training and predicting procedures. Also, it will help promote several works in practice, such as the implementation of embedded systems. Finally, a practical test, called the vehicular road test, is utilized to guarantee the level of confidence in the improvement of our algorithm.

An ultra-wideband (UWB) positioning system consists of at least three anchors and a tag for the positioning procedure. Via the UWB transceivers mounted on all devices in the system, we can obtain the distance information between each pair of devices and further realize the tag localization. However, the uncertain measurement in the real world may introduce incorrect measurement information, e.g., time, distance, positioning, and so on. Therefore, we intend to incorporate the technique of ensemble learning with UWB positioning to improve its performance. In this paper, we present two methods. The experimental results show that our ideas can be applied to different scenarios and work well. Of note, compared with the existing research in the literature, our first algorithm was more accurate and stable. Further, our second algorithm possessed even better performance than the first. Moreover, we also provide a comprehensive discussion for an ill-advised point, which is often used to evaluate the positioning efficiency in the literature.

Recently, many artificial intelligence applications in smart cars have been utilized in real life. Making an unmanned ground vehicle (UGV) capable of moving autonomously has become a critical topic. Hence, in this work, a novel method for a UGV to realize path planning and obstacle avoidance is presented using a deep deterministic policy gradient approach (DDPG). More specifically, the lidar sensor mounted on the vehicle is utilized to measure the distance between the vehicle and the surrounding obstacles, and the odometer measures the mileage of the vehicle for the purpose of estimating the current location. Then, the above sensed data are treated as the training data for the DDPG training procedure, and several experiments are performed in different settings utilizing the robot operating system (ROS) and the Gazebo simulator with a real robot module, TurtleBot3, to present a comprehensive discussion. The simulation results show that using the presented design and reward architecture, the DDPG method is better than the classic deep Q-network (DQN) method, e.g., taking fewer steps to reach the goal, less training time to find the smallest number of steps for reaching the goal, and so on.

Recently, autonomous driving has become one of the most popular topics for smart vehicles. However, traditional control strategies are mostly rule-based, which have poor adaptability to the time-varying traffic conditions. Similarly, they have difficulty coping with unexpected situations that may occur any time in the real-world environment. Hence, in this paper, we exploited Deep Reinforcement Learning (DRL) to enhance the quality and safety of autonomous driving control. Based on the road scenes and self-driving simulation modules provided by AirSim, we used the Deep Deterministic Policy Gradient (DDPG) and Recurrent Deterministic Policy Gradient (RDPG) algorithms, combined with the Convolutional Neural Network (CNN), to realize the autonomous driving control of self-driving cars. In particular, by using the real-time images of the road provided by AirSim as the training data, we carefully formulated an appropriate reward-generation method to improve the convergence speed of the adopted DDPG and RDPG models and the control performance of moving driverless cars.