Model and Algorithms for Classifying Anomalous Phenomena based on the Convergence of Acoustic-Visual Signals
Top Cited Papers
Open Access
- 11 January 2026
- journal article
- Published by Digital Technologies and Artificial Intelligence Development Research Institute in Problems of Computational and Applied Mathematics
- No. 6(70),p. 88-102
Abstract
This paper proposes a Context-adaptive Audio-Visual Neural Network (CAVN) model for anomaly detection in public safety systems. Existing approaches primarily rely on visual data and employ simple fusion strategies for combining modalities, which leads to limitations in capturing complex semantic relationships. The proposed model consists of four main components: a visual feature extraction module based on SlowFast architecture, an audio feature extraction module based on Audio Spectrogram Transformer (AST), a fusion module based on bidirectional cross-attention mechanism, and a temporal context aggregation module based on Transformer encoder. The main scientific novelty of the model lies in the adaptive modality balancing mechanism, which dynamically adjusts the relative importance of modalities under different conditions (dark/bright, noisy/quiet). Experimental results demonstrate that the proposed CAVN model outperforms existing methods by in overall accuracy and by in dark conditions. Ablation studies confirmed the contribution of each module to the overall performance.Keywords
This publication has 15 references indexed in Scilit:
- Проектирование и анализ системы солнечного водоснабжения для многоэтажных жилых зданий на основе булева программированияProblems of Computational and Applied Mathematics, 2026
- Regularization of the Initial-Boundary Value Problem for a Inhomogeneous Parabolic Equation with Changing Time DirectionProblems of Computational and Applied Mathematics, 2026
- Численное моделирование связанной динамической задачи термоупругости в напряженияхProblems of Computational and Applied Mathematics, 2026
- Three-Dimensional Mathematical Model and Numerical Solution Algorithm for Monitoring and Predicting In-Situ Leaching Processes in Porous MediumProblems of Computational and Applied Mathematics, 2026
- Model and Algorithms for Classifying Anomalous Phenomena based on the Convergence of Acoustic-Visual SignalsProblems of Computational and Applied Mathematics, 2026
- Моделирование пространственно-временной динамики площади водоёма (на примере Каттакурганского водохранилища) на основе индексов NDWI, NDVI, EVI и ансамблевых методов обученияProblems of Computational and Applied Mathematics, 2026
- MAVAD: Audio-Visual Dataset and Method for Anomaly Detection in Traffic Videos2024
- SSAST: Self-Supervised Audio Spectrogram TransformerProceedings of the AAAI Conference on Artificial Intelligence, 2022
- SlowFast Networks for Video Recognition2019
- Multimodal Machine Learning: A Survey and TaxonomyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018