Rua Yahya Aburasain
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Rua Yahya Aburasain has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Junaid Khan
Department of Environmental IT Engineering, Chungnam National University, Daejeon, South Korea
3 shared publications - Kyungsup Kim
Department of Computer Engineering, Chungnam National University, Daejeon, South Korea
3 shared publications - Awatef Salem Balobaid
Jazan University
2 shared publications Awatef Salim Balobaid
Department of Computer Science, College of Engineering and Computer Science, Jazan University, Jazan, Saudi Arabia
2 shared publicationsEunkyu Lee
Department of Computer Engineering, Chungnam National University, Daejeon, South Korea
2 shared publicationsUmar Zaman
Department of Computer Engineering, Chungnam National University, Daejeon, South Korea
2 shared publications
Latest Publications
- 17 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 72 (1) , 1230-1240
A Digital Twin-Driven Deep Reinforcement Learning Framework for End-to-End Mixed Traffic Scheduling
For end-to-end transmission of mixed traffic flows in IoT-enabled scenarios such as railway operation and maintenance, we propose a deterministic transmission scheduling mechanism based on deep reinforcement learning, termed the End-to-End Digital Twin-based Scheduling Mechanism for Online Mixed-Traffic (E2DTSM-OMT). In this work, digital twin technology enables high-fidelity mapping of the physical domain into its virtual representation. The proposed mechanism adopts differentiated scheduling strategies by classifying flows into three categories: monitoring and data aggregation, control and execution, and data analysis and optimization. Deterministic networking techniques are applied to ensure cross-domain, low-latency end-to-end communication. Through model and behavior mapping, the system constructs a mixed-traffic topology in the virtual space, where data transmission paths and time-slot resources are pre-allocated to minimize scheduling conflicts and resource contention. Additionally, a deep reinforcement learning (DRL) agent performs online decision-making to balance efficiency and effectiveness in dynamic environments. Compared with existing approaches, the integration of digital twins facilitates adaptive interaction between the physical and virtual worlds, supports DRL under non-stationary communication conditions, and prevents service degradation caused by real-world exploration. Simulation results verify that the proposed mechanism achieves reliable scheduling of mixed traffic flows, delivering higher transmission efficiency with reduced overall end-to-end latency.
- 17 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 72 (1) , 1230-1240
A Digital Twin-Driven Deep Reinforcement Learning Framework for End-to-End Mixed Traffic Scheduling
For end-to-end transmission of mixed traffic flows in IoT-enabled scenarios such as railway operation and maintenance, we propose a deterministic transmission scheduling mechanism based on deep reinforcement learning, termed the End-to-End Digital Twin-based Scheduling Mechanism for Online Mixed-Traffic (E2DTSM-OMT). In this work, digital twin technology enables high-fidelity mapping of the physical domain into its virtual representation. The proposed mechanism adopts differentiated scheduling strategies by classifying flows into three categories: monitoring and data aggregation, control and execution, and data analysis and optimization. Deterministic networking techniques are applied to ensure cross-domain, low-latency end-to-end communication. Through model and behavior mapping, the system constructs a mixed-traffic topology in the virtual space, where data transmission paths and time-slot resources are pre-allocated to minimize scheduling conflicts and resource contention. Additionally, a deep reinforcement learning (DRL) agent performs online decision-making to balance efficiency and effectiveness in dynamic environments. Compared with existing approaches, the integration of digital twins facilitates adaptive interaction between the physical and virtual worlds, supports DRL under non-stationary communication conditions, and prevents service degradation caused by real-world exploration. Simulation results verify that the proposed mechanism achieves reliable scheduling of mixed traffic flows, delivering higher transmission efficiency with reduced overall end-to-end latency.
- Open Access
- 1 October 2025
- Published by PeerJ in PeerJ Computer Science
- Vol. 11, e3224
Enhanced congestion prediction of traffic flow using a hybrid attention-based deep learning model
Traffic congestion has become a critical issue worldwide, significantly impacting the quality of life and causing economic losses in urban areas. To address this, the concept of smart cities has emerged, utilizing technologies like artificial intelligence (AI), machine learning (ML), and the Internet of Things (IoT) to optimize transportation systems. One of the main challenges in smart city traffic management is accurately predicting traffic congestion. Existing approaches often struggle with providing reliable predictions, especially in cloud-based environments, which can lead to inaccurate traffic estimates. This article, propose a novel data-driven approach to predict traffic congestion in smart cities using a hybrid model that combines bi-directional long short-term memory (Bi-LSTM), convolutional neural networks (CNN), and an attention network. This model leverages factors such as traffic conditions, road types, and weather data to improve prediction accuracy. The model’s performance is evaluated using key metrics including root mean squared error (RMSE), mean squared error (MSE), and mean absolute error (MAE), across four junctions (J1, J2, J3, J4). The proposed model consistently outperforms existing methods, including gated recurrent unit (GRU), long short-term memory (LSTM), CNN, and multi-layer perceptron (MLP). In Junction 1, the proposed model achieved the best results with RMSE = 0.252, MSE = 0.063, and MAE = 0.178. In Junction 2, the proposed model again led with RMSE = 0.561, while other models, like GRU and LSTM, showed higher error rates. These results confirm that the hybrid model is more effective at predicting traffic congestion, particularly in dynamic urban environments. The proposed approach offers a promising solution for improving traffic flow and optimizing smart city infrastructure.
- 1 October 2025
- Published by PeerJ in PeerJ Computer Science
- Vol. 11, e3224
Open AccessEnhanced congestion prediction of traffic flow using a hybrid attention-based deep learning model
Traffic congestion has become a critical issue worldwide, significantly impacting the quality of life and causing economic losses in urban areas. To address this, the concept of smart cities has emerged, utilizing technologies like artificial intelligence (AI), machine learning (ML), and the Internet of Things (IoT) to optimize transportation systems. One of the main challenges in smart city traffic management is accurately predicting traffic congestion. Existing approaches often struggle with providing reliable predictions, especially in cloud-based environments, which can lead to inaccurate traffic estimates. This article, propose a novel data-driven approach to predict traffic congestion in smart cities using a hybrid model that combines bi-directional long short-term memory (Bi-LSTM), convolutional neural networks (CNN), and an attention network. This model leverages factors such as traffic conditions, road types, and weather data to improve prediction accuracy. The model’s performance is evaluated using key metrics including root mean squared error (RMSE), mean squared error (MSE), and mean absolute error (MAE), across four junctions (J1, J2, J3, J4). The proposed model consistently outperforms existing methods, including gated recurrent unit (GRU), long short-term memory (LSTM), CNN, and multi-layer perceptron (MLP). In Junction 1, the proposed model achieved the best results with RMSE = 0.252, MSE = 0.063, and MAE = 0.178. In Junction 2, the proposed model again led with RMSE = 0.561, while other models, like GRU and LSTM, showed higher error rates. These results confirm that the hybrid model is more effective at predicting traffic congestion, particularly in dynamic urban environments. The proposed approach offers a promising solution for improving traffic flow and optimizing smart city infrastructure.
- Open Access
- 24 October 2024
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 19 (10) , e0310385
Deep learning innovations in South Korean maritime navigation: Enhancing vessel trajectories prediction with AIS data
Predicting ship trajectories can effectively forecast navigation trends and enable the orderly management of ships, which holds immense significance for maritime traffic safety. This paper introduces a novel ship trajectory prediction method utilizing Convolutional Neural Network (CNN), Deep Neural Network (DNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). Our research comprises two main parts: the first involves preprocessing the large raw AIS dataset to extract features, and the second focuses on trajectory prediction. We emphasize a specialized preprocessing approach tailored for AIS data, including advanced filtering techniques to remove outliers and erroneous data points, and the incorporation of contextual information such as environmental conditions and ship-specific characteristics. Our deep learning models utilize trajectory data sourced from the Automatic Identification System (AIS) to train and learn regular patterns within ship trajectory data, enabling them to predict trajectories for the next hour. Experimental results reveal that CNN has substantially reduced the Mean Absolute Error (MAE) and Mean Square Error (MSE) of ship trajectory prediction, showcasing superior performance compared to other deep learning algorithms. Additionally, a comparative analysis with other models—Recurrent Neural Network (RNN), GRU, LSTM, and DBS-LSTM—using metrics such as Average Displacement Error (ADE), Final Displacement Error (FDE), and Non-Linear ADE (NL-ADE), demonstrates our method’s robustness and accuracy. Our approach not only cleans the data but also enriches it, providing a robust foundation for subsequent deep learning applications in ship trajectory prediction. This improvement effectively enhances the accuracy of trajectory prediction, promising advancements in maritime traffic safety.
- 24 October 2024
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 19 (10) , e0310385
Open AccessDeep learning innovations in South Korean maritime navigation: Enhancing vessel trajectories prediction with AIS data
Predicting ship trajectories can effectively forecast navigation trends and enable the orderly management of ships, which holds immense significance for maritime traffic safety. This paper introduces a novel ship trajectory prediction method utilizing Convolutional Neural Network (CNN), Deep Neural Network (DNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). Our research comprises two main parts: the first involves preprocessing the large raw AIS dataset to extract features, and the second focuses on trajectory prediction. We emphasize a specialized preprocessing approach tailored for AIS data, including advanced filtering techniques to remove outliers and erroneous data points, and the incorporation of contextual information such as environmental conditions and ship-specific characteristics. Our deep learning models utilize trajectory data sourced from the Automatic Identification System (AIS) to train and learn regular patterns within ship trajectory data, enabling them to predict trajectories for the next hour. Experimental results reveal that CNN has substantially reduced the Mean Absolute Error (MAE) and Mean Square Error (MSE) of ship trajectory prediction, showcasing superior performance compared to other deep learning algorithms. Additionally, a comparative analysis with other models—Recurrent Neural Network (RNN), GRU, LSTM, and DBS-LSTM—using metrics such as Average Displacement Error (ADE), Final Displacement Error (FDE), and Non-Linear ADE (NL-ADE), demonstrates our method’s robustness and accuracy. Our approach not only cleans the data but also enriches it, providing a robust foundation for subsequent deep learning applications in ship trajectory prediction. This improvement effectively enhances the accuracy of trajectory prediction, promising advancements in maritime traffic safety.
- 22 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Hybrid Deep Learning with Optimized Hyperparameters Based Intrusion Detection in Internet of Things for Smart Farming
In addition to improving the country's economic prosperity, agriculture has significantly increased human life. Because of its rapid expansion, smart agriculture has recently gained popularity. This encompasses all computing technologies, including wireless sensor networks (WSN) and the Internet of Things (IoT). Agricultural regions strategically place IoT sensors to gather vital data about crops and fields, ultimately leading to higher overall production rates. To protect agricultural systems from cyber threats, intrusion detection in IoT-based smart farming uses deep learning (DL). DL systems have the ability to independently identify abnormal or unauthorized actions. This study creates a hybrid DL-enabled intrusion detection with particle swarm hyperparameter optimization (HDLID-PSHO) method that can be used in smart farming that is based on the IoT. The given HDLID-PSHO method carries out both the feature selection procedure and preprocessing. The proposed method uses a hybrid DL and transfer learning classification model to identify intrusion detection attempts. Finally, the Particle Swarm Optimization (PSO) algorithm allows for hyperparameter adjustment. The experimental validation confirmed that the proposed model performed well on ToN-IoT and NSL-KDD datasets.
- 22 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Hybrid Deep Learning with Optimized Hyperparameters Based Intrusion Detection in Internet of Things for Smart Farming
In addition to improving the country's economic prosperity, agriculture has significantly increased human life. Because of its rapid expansion, smart agriculture has recently gained popularity. This encompasses all computing technologies, including wireless sensor networks (WSN) and the Internet of Things (IoT). Agricultural regions strategically place IoT sensors to gather vital data about crops and fields, ultimately leading to higher overall production rates. To protect agricultural systems from cyber threats, intrusion detection in IoT-based smart farming uses deep learning (DL). DL systems have the ability to independently identify abnormal or unauthorized actions. This study creates a hybrid DL-enabled intrusion detection with particle swarm hyperparameter optimization (HDLID-PSHO) method that can be used in smart farming that is based on the IoT. The given HDLID-PSHO method carries out both the feature selection procedure and preprocessing. The proposed method uses a hybrid DL and transfer learning classification model to identify intrusion detection attempts. Finally, the Particle Swarm Optimization (PSO) algorithm allows for hyperparameter adjustment. The experimental validation confirmed that the proposed model performed well on ToN-IoT and NSL-KDD datasets.
- Open Access
- 16 October 2024
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 19 (10) , e0311734
Optimizing prediction accuracy in dynamic systems through neural network integration with Kalman and alpha-beta filters
In the realm of dynamic system analysis, the Kalman filter and the alpha-beta filter are widely recognized for their tracking and prediction capabilities. However, their performance is often limited by static parameters that cannot adapt to changing conditions. Addressing this limitation, this paper introduces innovative neural network-based prediction models that enhance the adaptability and accuracy of these conventional filters. Our approach involves the integration of neural networks within the filtering algorithms, enabling the dynamic augmentation of parameters in response to performance feedback. We present two modified filters: a neural network-based Kalman filter and an alpha-beta filter, each augmented to incorporate neural network-driven parameter tuning. The alpha-beta filter is enhanced with neural network outputs for its α and β parameters, while the Kalman filter employs a neural network to optimize its internal parameter R and noise factor F. We assess these advanced models using the root mean square error (RMSE) metric, where our neural network-based alpha-beta filter demonstrates a significant 38.2% improvement in prediction accuracy, and the neural network-based Kalman filter achieves a 53.4% enhancement. Hence, our novel approach of integrating neural networks into filtering algorithms stands out as an effective strategy to significantly enhance their performance in dynamic environments.
- 16 October 2024
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 19 (10) , e0311734
Open AccessOptimizing prediction accuracy in dynamic systems through neural network integration with Kalman and alpha-beta filters
In the realm of dynamic system analysis, the Kalman filter and the alpha-beta filter are widely recognized for their tracking and prediction capabilities. However, their performance is often limited by static parameters that cannot adapt to changing conditions. Addressing this limitation, this paper introduces innovative neural network-based prediction models that enhance the adaptability and accuracy of these conventional filters. Our approach involves the integration of neural networks within the filtering algorithms, enabling the dynamic augmentation of parameters in response to performance feedback. We present two modified filters: a neural network-based Kalman filter and an alpha-beta filter, each augmented to incorporate neural network-driven parameter tuning. The alpha-beta filter is enhanced with neural network outputs for its α and β parameters, while the Kalman filter employs a neural network to optimize its internal parameter R and noise factor F. We assess these advanced models using the root mean square error (RMSE) metric, where our neural network-based alpha-beta filter demonstrates a significant 38.2% improvement in prediction accuracy, and the neural network-based Kalman filter achieves a 53.4% enhancement. Hence, our novel approach of integrating neural networks into filtering algorithms stands out as an effective strategy to significantly enhance their performance in dynamic environments.
- 27 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Esophageal Cancer Classification in Initial Stages Using Deep and Transfer Learning
The sixth most frequent cancer and one with the highest death rate is esophageal cancer (EC). Initial diagnosis might increase the survival level of people with esophageal abnormalities. Esophagitis to non-dysplasia is the progression of EC in that order. Convolutional neural networks (CNN) equipped with effective feature extractors improve the prognosis $\text{of Barrett's}$ esophagus, esophagitis as well as precancerous stage conditions. More relevant characteristics may be extracted for the mechanized Barrett's esophagus classification as well as esophagitis using transfer learning methods with CNN. This research reports on a study on the employment of Enhanced Convolution Neural Networks (ECNN) for the classification of esophagitis. The evaluation $\text{will}$ be done with the K vasir open-access dataset, which delivers considered esophagitis along with Barrett's esophagus images. Several performance metrics, including Accuracy (acc), precision (prec), Recall (rec), and flscore, are used to assess the outcomes. To demonstrate the effectiveness of the proposed strategy, comparison studies are conducted with numerous different existing methodologies.- 27 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Esophageal Cancer Classification in Initial Stages Using Deep and Transfer Learning
The sixth most frequent cancer and one with the highest death rate is esophageal cancer (EC). Initial diagnosis might increase the survival level of people with esophageal abnormalities. Esophagitis to non-dysplasia is the progression of EC in that order. Convolutional neural networks (CNN) equipped with effective feature extractors improve the prognosis $\text{of Barrett's}$ esophagus, esophagitis as well as precancerous stage conditions. More relevant characteristics may be extracted for the mechanized Barrett's esophagus classification as well as esophagitis using transfer learning methods with CNN. This research reports on a study on the employment of Enhanced Convolution Neural Networks (ECNN) for the classification of esophagitis. The evaluation $\text{will}$ be done with the K vasir open-access dataset, which delivers considered esophagitis along with Barrett's esophagus images. Several performance metrics, including Accuracy (acc), precision (prec), Recall (rec), and flscore, are used to assess the outcomes. To demonstrate the effectiveness of the proposed strategy, comparison studies are conducted with numerous different existing methodologies.- Open Access
- 26 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 16621-16631
Enhanced Black Widow Optimization With Hybrid Deep Learning Enabled Intrusion Detection in Internet of Things-Based Smart Farming
Agriculture has played a significant role in the lifespan of human beings for their survival and the better economic growth of the country. Due to its intensified growth, smart agriculture has been a popular domain recently. This aggregates the benefits of any computing technologies, like wireless sensor networks (WSN), drones, Internet of Things (IoT), remote sensing, etc. The IoT system places sensors on agricultural areas to accumulate critical information for the crops and fields to increase the overall rate of productivity. While broadcasting the sensed information in the domains to the target, there is a chance of the existence of cyber-threats, which intruders design to attain access to the content being transferred. Intrusion detection in IoT-based smart farming using deep learning (DL) leverages the power of deep neural networks (DNN) to safeguard agricultural systems from cyber threats. DL algorithms can autonomously detect unauthorized and anomalies activities by analyzing data streams from sensors, IoT devices, and farm management systems. With this motivation, this study designs an Enhanced Black Widow Optimization with Hybrid Deep Learning Enabled Intrusion Detection (EBWO-HDLID) technique in the IoT-based Smart Farming environment. The proposed EBWO-HDLID technique captures complex patterns and detects significant intrusions, assuring the security and reliability of smart farming. In the presented EBWO-HDLID approach, the bald eagle search (BES) algorithm can be used for the feature selection process. For the intrusion detection and classification process, the EBWO-HDLID technique applies the HDL classification model. Finally, the EBWO algorithm can be used for the parameter tuning of the HDL technique. The experimental validation process demonstrates the satisfactory performance of the EBWO-HDLID model on two benchmark datasets: ToN-IoT, and Edge-IIoTset datasets.
- 26 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 16621-16631
Open AccessEnhanced Black Widow Optimization With Hybrid Deep Learning Enabled Intrusion Detection in Internet of Things-Based Smart Farming
Agriculture has played a significant role in the lifespan of human beings for their survival and the better economic growth of the country. Due to its intensified growth, smart agriculture has been a popular domain recently. This aggregates the benefits of any computing technologies, like wireless sensor networks (WSN), drones, Internet of Things (IoT), remote sensing, etc. The IoT system places sensors on agricultural areas to accumulate critical information for the crops and fields to increase the overall rate of productivity. While broadcasting the sensed information in the domains to the target, there is a chance of the existence of cyber-threats, which intruders design to attain access to the content being transferred. Intrusion detection in IoT-based smart farming using deep learning (DL) leverages the power of deep neural networks (DNN) to safeguard agricultural systems from cyber threats. DL algorithms can autonomously detect unauthorized and anomalies activities by analyzing data streams from sensors, IoT devices, and farm management systems. With this motivation, this study designs an Enhanced Black Widow Optimization with Hybrid Deep Learning Enabled Intrusion Detection (EBWO-HDLID) technique in the IoT-based Smart Farming environment. The proposed EBWO-HDLID technique captures complex patterns and detects significant intrusions, assuring the security and reliability of smart farming. In the presented EBWO-HDLID approach, the bald eagle search (BES) algorithm can be used for the feature selection process. For the intrusion detection and classification process, the EBWO-HDLID technique applies the HDL classification model. Finally, the EBWO algorithm can be used for the parameter tuning of the HDL technique. The experimental validation process demonstrates the satisfactory performance of the EBWO-HDLID model on two benchmark datasets: ToN-IoT, and Edge-IIoTset datasets.
- Open Access
- 17 January 2024
- Published by MDPI AG in Preprints.org
A Hybrid Machine Learning and Optimization Algorithm for Enhanced User Comfort and Energy Efficiency in Smart Homes
Researchers are always looking to find ways to use energy more efficiently while keeping people comfortable at home. They've tried different methods like fuzzy logic and genetic algorithms to do this. In this paper, we introduce a novel model that combines different techniques like data smoothing, optimization, machine learning, and control algorithms to reach this goal in smart homes. The proposed model is made up of three main parts: smoothing, optimization, and control. The smoothing part uses the Kalman filter to get rid of any unnecessary noise and make sure the data used to predict energy use is clean and smooth. The optimization part uses both genetic with firefly algorithms and artificial neural networks to minimize the gap between the actual environmental conditions and the conditions users find comfortable. It adjusts the comfort settings in real-time using machine learning. The control part uses Mamdani fuzzy logic to make sure energy is delivered in the best way possible to the systems that control temperature, lighting, etc., making the home comfortable and saving energy at the same time. Comparative analyses demonstrate the superiority of the proposed model over existing algorithms, including the Particle Swarm Optimization (PSO) algorithm. Additionally, the paper discusses the importance of adaptive controllers in addressing issues associated with incorrect PID controller selection and highlights the benefits of using the FA-GA in energy optimization and comfort improvement. Overall, the proposed model offers an effective solution for efficient energy consumption optimization and enhanced user comfort in diverse settings through the integration of advanced algorithms and optimization modules.
- 17 January 2024
- Published by MDPI AG in Preprints.org
Open AccessA Hybrid Machine Learning and Optimization Algorithm for Enhanced User Comfort and Energy Efficiency in Smart Homes
Researchers are always looking to find ways to use energy more efficiently while keeping people comfortable at home. They've tried different methods like fuzzy logic and genetic algorithms to do this. In this paper, we introduce a novel model that combines different techniques like data smoothing, optimization, machine learning, and control algorithms to reach this goal in smart homes. The proposed model is made up of three main parts: smoothing, optimization, and control. The smoothing part uses the Kalman filter to get rid of any unnecessary noise and make sure the data used to predict energy use is clean and smooth. The optimization part uses both genetic with firefly algorithms and artificial neural networks to minimize the gap between the actual environmental conditions and the conditions users find comfortable. It adjusts the comfort settings in real-time using machine learning. The control part uses Mamdani fuzzy logic to make sure energy is delivered in the best way possible to the systems that control temperature, lighting, etc., making the home comfortable and saving energy at the same time. Comparative analyses demonstrate the superiority of the proposed model over existing algorithms, including the Particle Swarm Optimization (PSO) algorithm. Additionally, the paper discusses the importance of adaptive controllers in addressing issues associated with incorrect PID controller selection and highlights the benefits of using the FA-GA in energy optimization and comfort improvement. Overall, the proposed model offers an effective solution for efficient energy consumption optimization and enhanced user comfort in diverse settings through the integration of advanced algorithms and optimization modules.
- Open Access
- 27 July 2022
- Published by Springer Nature
- p. 12-33
A Coarse-to-Fine Multi-class Object Detection in Drone Images Using Convolutional Neural Networks
Multi-class object detection has a rapid evolution in the last few years with the rise of deep Convolutional Neural Networks (CNNs) learning based, in particular. However, the success approaches are based on high resolution ground level images and extremely large volume of data as in COCO and VOC datasets. On the other hand, the availability of the drones has been increased in the last few years and hence several new applications have been established. One of such is understanding drone footage by analysing, detecting, recognizing different objects in the covered area. In this study conducted, a collection of large images captured by a drone flying at a fixed altitude in a desert area located within the United Arab Emirates (UAE) is given and it is utilised for training and evaluating the CNN networks to be investigated. Three state-of-the-art CNN architectures, namely SSD-500 with VGGNet-16 meta-architecture, SSD-500 with ResNet meta-architecture and YOLO-V3 with Darknet-53 are optimally configured, re-trained, tested and evaluated for the detection of three different classes of objects in the captured footage, namely, palm trees, group-of-animals/cattle and animal sheds in farms. Our preliminary experiments revealed that YOLO-V3 outperformed SSD-500 with VGGNet-16 by a large margin and has a considerable improvement as compared to using SSD-500 with ResNet. Therefore, it has been selected for further investigation, aiming to propose an efficient coarse-to-fine object detection model for multi-class object detection in drone images. To this end, the impact of changing the activation function of the hidden units and the pooling type in the pooling layer has been investigated in detail. In addition, the impact of tuning the learning rate and the selection of the most effective optimization method for general hyper-parameters tuning is also investigated. The result demonstrated that the multi-class object detector developed has precision of 0.99, a recall of 0.94 and an F-score of 0.96, proving the efficiency of the multi-class object detection network developed.
- 27 July 2022
- Published by Springer Nature
- p. 12-33
Open AccessA Coarse-to-Fine Multi-class Object Detection in Drone Images Using Convolutional Neural Networks
Multi-class object detection has a rapid evolution in the last few years with the rise of deep Convolutional Neural Networks (CNNs) learning based, in particular. However, the success approaches are based on high resolution ground level images and extremely large volume of data as in COCO and VOC datasets. On the other hand, the availability of the drones has been increased in the last few years and hence several new applications have been established. One of such is understanding drone footage by analysing, detecting, recognizing different objects in the covered area. In this study conducted, a collection of large images captured by a drone flying at a fixed altitude in a desert area located within the United Arab Emirates (UAE) is given and it is utilised for training and evaluating the CNN networks to be investigated. Three state-of-the-art CNN architectures, namely SSD-500 with VGGNet-16 meta-architecture, SSD-500 with ResNet meta-architecture and YOLO-V3 with Darknet-53 are optimally configured, re-trained, tested and evaluated for the detection of three different classes of objects in the captured footage, namely, palm trees, group-of-animals/cattle and animal sheds in farms. Our preliminary experiments revealed that YOLO-V3 outperformed SSD-500 with VGGNet-16 by a large margin and has a considerable improvement as compared to using SSD-500 with ResNet. Therefore, it has been selected for further investigation, aiming to propose an efficient coarse-to-fine object detection model for multi-class object detection in drone images. To this end, the impact of changing the activation function of the hidden units and the pooling type in the pooling layer has been investigated in detail. In addition, the impact of tuning the learning rate and the selection of the most effective optimization method for general hyper-parameters tuning is also investigated. The result demonstrated that the multi-class object detector developed has precision of 0.99, a recall of 0.94 and an F-score of 0.96, proving the efficiency of the multi-class object detection network developed.