Ali Alqahtani
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Ali Alqahtani has a total of 60 co-authors. Here are the top co-authors by number of shared publications:
- Kian Ming Lim
University of Nottingham Ningbo China
14 shared publications - Chin Poo Lee
University of Nottingham Ningbo China
13 shared publications - Mohammed Ali
Department of Computer Science, Applied College, King Khalid University, Abha, Saudi Arabia
7 shared publications - Mark W. Jones
Department of Computer Science, Swansea University, Swansea, UK
3 shared publications - Mugahed A. Al-Antari
Department of Artificial Intelligence and Data Science, College of AI Convergence, Daeyang AI Center, Sejong University, Seoul, Republic of Korea
3 shared publications - Xianghua Xie
Department of Computer Science, Swansea University, Swansea, UK
3 shared publications
Latest Publications
- Open Access
- 28 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 16034-16050
Ensemble Artificial Intelligence Framework for Real-Time Automatic Vehicle Detection and Classifications
Vehicle detection plays a vital role in a variety of applications such as autonomous driving, traffic management, and public safety. In this article, we propose an ensemble artificial intelligence framework to simultaneously detect and classify various vehicles on public roads. The framework aims to improve the detection accuracy of vehicles and their small targets at long distances in different environmental conditions: rainy, foggy, and night scenes. To achieve this goal, we first construct a benchmark vehicle dataset (Dataset 4) containing multiple vehicle types (Truck, Van, Car, Bus and Two-wheelers) across complex environmental conditions. Subsequently, we adopt multiple state-of-the-art AI detectors, including finetuned versions of YOLOv5, YOLOv7, YOLOv8, YOLOv10, DETR-ResNet50, and Faster R-CNN. Using the random-search approach the hyperparameters of the AI models are automatically optimized to select and fuse the top three models into a unified detection framework through an ensemble learning strategy. The ensemble framework aims to combine the advantages of multiple detectors to improve the detection accuracy and robustness under different vehicle types and conditions. Our proposed integrated model framework based on YOLOv7, YOLOv8 and YOLOv10 further improves the model performance using non-maximum suppression (NMS) and dynamic weighting methods, outperforming single detectors with excellent detection performance of 94.2% precision, 94.0% recall, 94.1% F1-score, and 96.6% mAP. The main contribution is the proposal of a novel vehicle detection integration framework and a comprehensive benchmark dataset to address the challenges faced by current vehicle detection systems. The system is deployed to test real driving videos, and the experimental results demonstrate the practical applicability of our proposed method.
- 28 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 16034-16050
Open AccessEnsemble Artificial Intelligence Framework for Real-Time Automatic Vehicle Detection and Classifications
Vehicle detection plays a vital role in a variety of applications such as autonomous driving, traffic management, and public safety. In this article, we propose an ensemble artificial intelligence framework to simultaneously detect and classify various vehicles on public roads. The framework aims to improve the detection accuracy of vehicles and their small targets at long distances in different environmental conditions: rainy, foggy, and night scenes. To achieve this goal, we first construct a benchmark vehicle dataset (Dataset 4) containing multiple vehicle types (Truck, Van, Car, Bus and Two-wheelers) across complex environmental conditions. Subsequently, we adopt multiple state-of-the-art AI detectors, including finetuned versions of YOLOv5, YOLOv7, YOLOv8, YOLOv10, DETR-ResNet50, and Faster R-CNN. Using the random-search approach the hyperparameters of the AI models are automatically optimized to select and fuse the top three models into a unified detection framework through an ensemble learning strategy. The ensemble framework aims to combine the advantages of multiple detectors to improve the detection accuracy and robustness under different vehicle types and conditions. Our proposed integrated model framework based on YOLOv7, YOLOv8 and YOLOv10 further improves the model performance using non-maximum suppression (NMS) and dynamic weighting methods, outperforming single detectors with excellent detection performance of 94.2% precision, 94.0% recall, 94.1% F1-score, and 96.6% mAP. The main contribution is the proposal of a novel vehicle detection integration framework and a comprehensive benchmark dataset to address the challenges faced by current vehicle detection systems. The system is deployed to test real driving videos, and the experimental results demonstrate the practical applicability of our proposed method.
- Open Access
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 147130-147176
A Survey on Vision Transformers: Design Pressures, Architectural Taxonomy, and Future Outlook
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 147130-147176
Open AccessA Survey on Vision Transformers: Design Pressures, Architectural Taxonomy, and Future Outlook
- Open Access
- 12 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 160722-160736
QARR-QGF: A Dual-Module Data Augmentation Framework for Enhanced Few-Shot Question Answering
Few-shot question answering (QA) aims to train effective QA models using very limited annotated data, but existing approaches often struggle to generalize due to insufficient training examples. This paper proposes a two-stage data augmentation framework, called Question-Answer Replacement and Removal and Question Generation and Filtering (QARR-QGF), to address this challenge. The QARR module enhances pretraining data by systematically replacing and removing question-answer pairs to create more diverse examples. During fine-tuning, the QGF module generates paraphrased questions and applies semantic filtering to retain high-quality training samples. The framework is evaluated using three widely used generative models: Longformer-Encoder-Decoder (LED), BART, and T5, on the SQuAD, HotpotQA, and Natural Questions datasets. Experimental results show that QARR-QGF consistently improves performance across all datasets and few-shot settings. For example, the QARR-QGF-T5 model achieves F1 scores of 82.3% on SQuAD, 59.9% on HotpotQA, and 59.0% on Natural Questions in the 16-shot setting, outperforming previous state-of-the-art methods. These results demonstrate the effectiveness of QARR-QGF in improving few-shot QA performance by generating richer and more diverse training data.
- 12 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 160722-160736
Open AccessQARR-QGF: A Dual-Module Data Augmentation Framework for Enhanced Few-Shot Question Answering
Few-shot question answering (QA) aims to train effective QA models using very limited annotated data, but existing approaches often struggle to generalize due to insufficient training examples. This paper proposes a two-stage data augmentation framework, called Question-Answer Replacement and Removal and Question Generation and Filtering (QARR-QGF), to address this challenge. The QARR module enhances pretraining data by systematically replacing and removing question-answer pairs to create more diverse examples. During fine-tuning, the QGF module generates paraphrased questions and applies semantic filtering to retain high-quality training samples. The framework is evaluated using three widely used generative models: Longformer-Encoder-Decoder (LED), BART, and T5, on the SQuAD, HotpotQA, and Natural Questions datasets. Experimental results show that QARR-QGF consistently improves performance across all datasets and few-shot settings. For example, the QARR-QGF-T5 model achieves F1 scores of 82.3% on SQuAD, 59.9% on HotpotQA, and 59.0% on Natural Questions in the 16-shot setting, outperforming previous state-of-the-art methods. These results demonstrate the effectiveness of QARR-QGF in improving few-shot QA performance by generating richer and more diverse training data.
- Open Access
- 14 August 2025
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-23
A graph attention network-based multi-agent reinforcement learning framework for robust detection of smart contract vulnerabilities
Smart contracts have revolutionized decentralized applications by automating agreement enforcement on blockchain platforms. However, detecting vulnerabilities in smart contract interactions remains challenging due to complex state interdependencies. This paper presents a novel approach using multi-agent Reinforcement Learning (MARL) to identify smart contract vulnerabilities. We integrate a Hierarchical Graph Attention Network (HGAT) into a Multi-Agent Actor-Critic framework, decomposing vulnerability detection into complementary policies: a high-level policy encoding historical interactions and a low-level policy capturing structured actions within contract state spaces. By modeling interactions as multistep reasoning paths, our MARL framework effectively navigates complex transaction sequences and resolves semantic ambiguities across different contract states. Experimental evaluations on real-world blockchain datasets demonstrate significant improvements in detecting multiple vulnerability types. For reentrancy attacks, our model achieves 93.8% accuracy and an 89.8% F1 score. The framework also performs strongly in detecting front running (88.9% accuracy), denial-of-service attacks (91.2% accuracy), and unchecked low-level vulnerabilities (91.6% accuracy), outperforming existing approaches across all vulnerability categories.
- 14 August 2025
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-23
Open AccessA graph attention network-based multi-agent reinforcement learning framework for robust detection of smart contract vulnerabilities
Smart contracts have revolutionized decentralized applications by automating agreement enforcement on blockchain platforms. However, detecting vulnerabilities in smart contract interactions remains challenging due to complex state interdependencies. This paper presents a novel approach using multi-agent Reinforcement Learning (MARL) to identify smart contract vulnerabilities. We integrate a Hierarchical Graph Attention Network (HGAT) into a Multi-Agent Actor-Critic framework, decomposing vulnerability detection into complementary policies: a high-level policy encoding historical interactions and a low-level policy capturing structured actions within contract state spaces. By modeling interactions as multistep reasoning paths, our MARL framework effectively navigates complex transaction sequences and resolves semantic ambiguities across different contract states. Experimental evaluations on real-world blockchain datasets demonstrate significant improvements in detecting multiple vulnerability types. For reentrancy attacks, our model achieves 93.8% accuracy and an 89.8% F1 score. The framework also performs strongly in detecting front running (88.9% accuracy), denial-of-service attacks (91.2% accuracy), and unchecked low-level vulnerabilities (91.6% accuracy), outperforming existing approaches across all vulnerability categories.
- Open Access
- 29 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 159280-159295
QARR-FSQA: Question-Answer Replacement and Removal Pretraining Framework for Few-Shot Question Answering
In Natural Language Processing, creating training data for question answering (QA) systems typically requires significant effort and expertise. This challenge is amplified in few-shot scenarios where only a limited number of training samples are available. This paper proposes a novel pretraining framework to enhance few-shot question answering (FSQA) capabilities. It begins with the selection of the Discrete Reasoning Over the Content of Paragraphs (DROP) dataset, designed for English reading comprehension tasks involving various reasoning types. Data preprocessing converts question-answer pairs into a predefined template, consisting of a concatenated sequence of the question, a mask token with a prefix, and the context, forming the input sequence, while the target sequence includes the question and answer. The Question-Answer Replacement and Removal (QARR) technique augments the dataset by integrating the answer into the question and selectively removing words. Various templates for question-answer pairs are introduced. Models like BART, T5, and LED are then used to evaluate the framework’s performance, undergoing further pretraining on the augmented dataset with their respective architectures and optimization objectives. The study also investigates the impact of different templates on model performance in few-shot QA tasks. Evaluated on three datasets in few-shot scenarios, the QARR-T5 method outperforms state-of-the-art FSQA techniques, achieving the highest F1 scores of 81.7% in 16-shot and 32-shot, 82.7% in 64-shot, and 84.5% in 128-shot on the SQuAD dataset. This demonstrates the framework’s effectiveness in improving models’ generalization and performance on new datasets with limited samples, advancing few-shot QA.
- 29 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 159280-159295
Open AccessQARR-FSQA: Question-Answer Replacement and Removal Pretraining Framework for Few-Shot Question Answering
In Natural Language Processing, creating training data for question answering (QA) systems typically requires significant effort and expertise. This challenge is amplified in few-shot scenarios where only a limited number of training samples are available. This paper proposes a novel pretraining framework to enhance few-shot question answering (FSQA) capabilities. It begins with the selection of the Discrete Reasoning Over the Content of Paragraphs (DROP) dataset, designed for English reading comprehension tasks involving various reasoning types. Data preprocessing converts question-answer pairs into a predefined template, consisting of a concatenated sequence of the question, a mask token with a prefix, and the context, forming the input sequence, while the target sequence includes the question and answer. The Question-Answer Replacement and Removal (QARR) technique augments the dataset by integrating the answer into the question and selectively removing words. Various templates for question-answer pairs are introduced. Models like BART, T5, and LED are then used to evaluate the framework’s performance, undergoing further pretraining on the augmented dataset with their respective architectures and optimization objectives. The study also investigates the impact of different templates on model performance in few-shot QA tasks. Evaluated on three datasets in few-shot scenarios, the QARR-T5 method outperforms state-of-the-art FSQA techniques, achieving the highest F1 scores of 81.7% in 16-shot and 32-shot, 82.7% in 64-shot, and 84.5% in 128-shot on the SQuAD dataset. This demonstrates the framework’s effectiveness in improving models’ generalization and performance on new datasets with limited samples, advancing few-shot QA.
- Open Access
- 30 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 79327-79341
PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition
Facial Expression Recognition (FER) is a machine learning problem that deals with recognizing human facial expressions. While existing work has achieved performance improvements in recent years, FER in the wild and under challenging conditions remains a challenge. In this paper, a lightweight patch and attention network based on MobileNetV1, referred to as PAtt-Lite, is proposed to improve FER performance under challenging conditions. A truncated ImageNet-pre-trained MobileNetV1 is utilized as the backbone feature extractor of the proposed method. In place of the truncated layers is a patch extraction block that is proposed for extracting significant local facial features to enhance the representation from MobileNetV1, especially under challenging conditions. An attention classifier is also proposed to improve the learning of these patched feature maps from the extremely lightweight feature extractor. The experimental results on public benchmark databases proved the effectiveness of the proposed method. PAtt-Lite achieved state-of-the-art results on CK+, RAF-DB, FER2013, FERPlus, and the challenging conditions subsets for RAF-DB and FERPlus.
- 30 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 79327-79341
Open AccessPAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition
Facial Expression Recognition (FER) is a machine learning problem that deals with recognizing human facial expressions. While existing work has achieved performance improvements in recent years, FER in the wild and under challenging conditions remains a challenge. In this paper, a lightweight patch and attention network based on MobileNetV1, referred to as PAtt-Lite, is proposed to improve FER performance under challenging conditions. A truncated ImageNet-pre-trained MobileNetV1 is utilized as the backbone feature extractor of the proposed method. In place of the truncated layers is a patch extraction block that is proposed for extracting significant local facial features to enhance the representation from MobileNetV1, especially under challenging conditions. An attention classifier is also proposed to improve the learning of these patched feature maps from the extremely lightweight feature extractor. The experimental results on public benchmark databases proved the effectiveness of the proposed method. PAtt-Lite achieved state-of-the-art results on CK+, RAF-DB, FER2013, FERPlus, and the challenging conditions subsets for RAF-DB and FERPlus.
- Open Access
- 31 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 18237-18250
MaxMViT-MLP: Multiaxis and Multiscale Vision Transformers Fusion Network for Speech Emotion Recognition
Vision Transformers, known for their innovative architectural design and modeling capabilities, have gained significant attention in computer vision. This paper presents a dual-path approach that leverages the strengths of the Multi-Axis Vision Transformer (MaxViT) and the Improved Multiscale Vision Transformer (MViTv2). It starts by encoding speech signals into Constant-Q Transform (CQT) spectrograms and Mel Spectrograms with Short-Time Fourier Transform (Mel-STFT). The CQT spectrogram is then fed into the MaxViT model, while the Mel-STFT is input to the MViTv2 model to extract informative features from the spectrograms. These features are integrated and passed into a Multilayer Perceptron (MLP) model for final classification. This hybrid model is named the “MaxViT and MViTv2 Fusion Network with Multilayer Perceptron (MaxMViT-MLP).” The MaxMViT-MLP model achieves remarkable results with an accuracy of 95.28% on the Emo-DB, 89.12% on the RAVDESS dataset, and 68.39% on the IEMOCAP dataset, substantiating the advantages of integrating multiple audio feature representations and Vision Transformers in speech emotion recognition.
- 31 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 18237-18250
Open AccessMaxMViT-MLP: Multiaxis and Multiscale Vision Transformers Fusion Network for Speech Emotion Recognition
Vision Transformers, known for their innovative architectural design and modeling capabilities, have gained significant attention in computer vision. This paper presents a dual-path approach that leverages the strengths of the Multi-Axis Vision Transformer (MaxViT) and the Improved Multiscale Vision Transformer (MViTv2). It starts by encoding speech signals into Constant-Q Transform (CQT) spectrograms and Mel Spectrograms with Short-Time Fourier Transform (Mel-STFT). The CQT spectrogram is then fed into the MaxViT model, while the Mel-STFT is input to the MViTv2 model to extract informative features from the spectrograms. These features are integrated and passed into a Multilayer Perceptron (MLP) model for final classification. This hybrid model is named the “MaxViT and MViTv2 Fusion Network with Multilayer Perceptron (MaxMViT-MLP).” The MaxMViT-MLP model achieves remarkable results with an accuracy of 95.28% on the Emo-DB, 89.12% on the RAVDESS dataset, and 68.39% on the IEMOCAP dataset, substantiating the advantages of integrating multiple audio feature representations and Vision Transformers in speech emotion recognition.
- Open Access
- 16 November 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (22) , 12384
A Review on Congestion Mitigation Techniques in Ultra-Dense Wireless Sensor Networks: State-of-the-Art Future Emerging Artificial Intelligence-Based Solutions
The Internet of Things (IoT) and wireless sensor networks (WSNs) have evolved rapidly due to technological breakthroughs. WSNs generate high traffic due to the growing number of sensor nodes. Congestion is one of several problems caused by the huge amount of data in WSNs. When wireless network resources are limited and IoT devices require more and more resources, congestion occurs in extremely dense WSN-based IoT networks. Reduced throughput, reduced network capacity, and reduced energy efficiency within WSNs are all effects of congestion. These consequences eventually lead to network outages due to underutilized network resources, increased network operating costs, and significantly degraded quality of service (QoS). Therefore, it is critical to deal with congestion in WSN-based IoT networks. Researchers have developed a number of approaches to address this problem, with new solutions based on artificial intelligence (AI) standing out. This research examines how new AI-based algorithms contribute to congestion mitigation in WSN-based IoT networks and the various congestion mitigation strategies that have helped reduce congestion. This study also highlights the limitations of AI-based solutions, including where and why they are used in WSNs, and a comparative study of the current literature that makes this study novel. The study concludes with a discussion of its significance and potential future study topics. The topic of congestion reduction in ultra-dense WSN-based IoT networks, as well as the current state of the art and emerging future solutions, demonstrates their significant expertise in reducing WSN congestion. These solutions contribute to network optimization, throughput enhancement, quality of service improvement, network capacity expansion, and overall WSN efficiency improvement.
- 16 November 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (22) , 12384
Open AccessA Review on Congestion Mitigation Techniques in Ultra-Dense Wireless Sensor Networks: State-of-the-Art Future Emerging Artificial Intelligence-Based Solutions
The Internet of Things (IoT) and wireless sensor networks (WSNs) have evolved rapidly due to technological breakthroughs. WSNs generate high traffic due to the growing number of sensor nodes. Congestion is one of several problems caused by the huge amount of data in WSNs. When wireless network resources are limited and IoT devices require more and more resources, congestion occurs in extremely dense WSN-based IoT networks. Reduced throughput, reduced network capacity, and reduced energy efficiency within WSNs are all effects of congestion. These consequences eventually lead to network outages due to underutilized network resources, increased network operating costs, and significantly degraded quality of service (QoS). Therefore, it is critical to deal with congestion in WSN-based IoT networks. Researchers have developed a number of approaches to address this problem, with new solutions based on artificial intelligence (AI) standing out. This research examines how new AI-based algorithms contribute to congestion mitigation in WSN-based IoT networks and the various congestion mitigation strategies that have helped reduce congestion. This study also highlights the limitations of AI-based solutions, including where and why they are used in WSNs, and a comparative study of the current literature that makes this study novel. The study concludes with a discussion of its significance and potential future study topics. The topic of congestion reduction in ultra-dense WSN-based IoT networks, as well as the current state of the art and emerging future solutions, demonstrates their significant expertise in reducing WSN congestion. These solutions contribute to network optimization, throughput enhancement, quality of service improvement, network capacity expansion, and overall WSN efficiency improvement.
- Open Access
- 10 November 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (22) , 12204
SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition
Hand gesture recognition (HGR) is a rapidly evolving field with the potential to revolutionize human–computer interactions by enabling machines to interpret and understand human gestures for intuitive communication and control. However, HGR faces challenges such as the high similarity of hand gestures, real-time performance, and model generalization. To address these challenges, this paper proposes the stacking of distilled vision transformers, referred to as SDViT, for hand gesture recognition. An initially pretrained vision transformer (ViT) featuring a self-attention mechanism is introduced to effectively capture intricate connections among image patches, thereby enhancing its capability to handle the challenge of high similarity between hand gestures. Subsequently, knowledge distillation is proposed to compress the ViT model and improve model generalization. Multiple distilled ViTs are then stacked to achieve higher predictive performance and reduce overfitting. The proposed SDViT model achieves a promising performance on three benchmark datasets for hand gesture recognition: the American Sign Language (ASL) dataset, the ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. The accuracies achieved on these datasets are 100.00%, 99.60%, and 100.00%, respectively.
- 10 November 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (22) , 12204
Open AccessSDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition
Hand gesture recognition (HGR) is a rapidly evolving field with the potential to revolutionize human–computer interactions by enabling machines to interpret and understand human gestures for intuitive communication and control. However, HGR faces challenges such as the high similarity of hand gestures, real-time performance, and model generalization. To address these challenges, this paper proposes the stacking of distilled vision transformers, referred to as SDViT, for hand gesture recognition. An initially pretrained vision transformer (ViT) featuring a self-attention mechanism is introduced to effectively capture intricate connections among image patches, thereby enhancing its capability to handle the challenge of high similarity between hand gestures. Subsequently, knowledge distillation is proposed to compress the ViT model and improve model generalization. Multiple distilled ViTs are then stacked to achieve higher predictive performance and reduce overfitting. The proposed SDViT model achieves a promising performance on three benchmark datasets for hand gesture recognition: the American Sign Language (ASL) dataset, the ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. The accuracies achieved on these datasets are 100.00%, 99.60%, and 100.00%, respectively.
- Open Access
EnViTSA: Ensemble of Vision Transformer with SpecAugment for Acoustic Event Classification
Recent successes in deep learning have inspired researchers to apply deep neural networks to Acoustic Event Classification (AEC). While deep learning methods can train effective AEC models, they are susceptible to overfitting due to the models’ high complexity. In this paper, we introduce EnViTSA, an innovative approach that tackles key challenges in AEC. EnViTSA combines an ensemble of Vision Transformers with SpecAugment, a novel data augmentation technique, to significantly enhance AEC performance. Raw acoustic signals are transformed into Log Mel-spectrograms using Short-Time Fourier Transform, resulting in a fixed-size spectrogram representation. To address data scarcity and overfitting issues, we employ SpecAugment to generate additional training samples through time masking and frequency masking. The core of EnViTSA resides in its ensemble of pre-trained Vision Transformers, harnessing the unique strengths of the Vision Transformer architecture. This ensemble approach not only reduces inductive biases but also effectively mitigates overfitting. In this study, we evaluate the EnViTSA method on three benchmark datasets: ESC-10, ESC-50, and UrbanSound8K. The experimental results underscore the efficacy of our approach, achieving impressive accuracy scores of 93.50%, 85.85%, and 83.20% on ESC-10, ESC-50, and UrbanSound8K, respectively. EnViTSA represents a substantial advancement in AEC, demonstrating the potential of Vision Transformers and SpecAugment in the acoustic domain.
Open AccessEnViTSA: Ensemble of Vision Transformer with SpecAugment for Acoustic Event Classification
Recent successes in deep learning have inspired researchers to apply deep neural networks to Acoustic Event Classification (AEC). While deep learning methods can train effective AEC models, they are susceptible to overfitting due to the models’ high complexity. In this paper, we introduce EnViTSA, an innovative approach that tackles key challenges in AEC. EnViTSA combines an ensemble of Vision Transformers with SpecAugment, a novel data augmentation technique, to significantly enhance AEC performance. Raw acoustic signals are transformed into Log Mel-spectrograms using Short-Time Fourier Transform, resulting in a fixed-size spectrogram representation. To address data scarcity and overfitting issues, we employ SpecAugment to generate additional training samples through time masking and frequency masking. The core of EnViTSA resides in its ensemble of pre-trained Vision Transformers, harnessing the unique strengths of the Vision Transformer architecture. This ensemble approach not only reduces inductive biases but also effectively mitigates overfitting. In this study, we evaluate the EnViTSA method on three benchmark datasets: ESC-10, ESC-50, and UrbanSound8K. The experimental results underscore the efficacy of our approach, achieving impressive accuracy scores of 93.50%, 85.85%, and 83.20% on ESC-10, ESC-50, and UrbanSound8K, respectively. EnViTSA represents a substantial advancement in AEC, demonstrating the potential of Vision Transformers and SpecAugment in the acoustic domain.