Parallel Bilingual Datasets: A Multimodal Deep Learning Framework for Proficiency and Style Classification
Open Access
- 30 April 2026
- journal article
- research article
- Published by MDPI AG in Multimodal Technologies and Interaction
- Vol. 10 (5) , 47
Abstract
This study presents a multimodal deep learning framework for automatic proficiency and style classification of parallel Bilingual Tamil–Hindi learner data. The proposed system employs a dual-headed neural architecture to simultaneously predict proficiency levels (Basic, Advanced) and stylistic categories (Formal, Literary) using shared feature representations. A curated dataset of bilingual text samples is utilized, along with synthetic speech generated through text-to-speech (TTS) to enable controlled multimodal experimentation. Five deep learning architectures are evaluated under text-only, audio-only, and learnable fusion settings. Experimental findings indicate that text-based models consistently achieve strong performance in both proficiency and style classification tasks. In contrast, the audio-only model demonstrates limited effectiveness, highlighting the constraints of synthetic acoustic features in capturing meaningful linguistic information. The fusion models provide only marginal improvements over text-based approaches, suggesting that textual representations play a dominant role in proficiency and stylistic classification within controlled datasets. These results emphasize the importance of linguistic features over acoustic signals for automated language assessment in low-resource settings. The proposed framework provides a scalable and reproducible approach and offers a foundation for future work incorporating real speech data and more diverse linguistic inputs.Keywords
This publication has 32 references indexed in Scilit:
- Real-time tracker of chicken for poultry based on attention mechanism-enhanced YOLO-Chicken algorithmComputers and Electronics in Agriculture, 2025
- Classification of human-written and AI-generated sentences using a hybrid CNN-GRU model optimized by the spotted hyena algorithmAlexandria Engineering Journal, 2025
- Choice of language in the construction of cultural identity by Tamil speakers in IndiaInternational Journal of Language and Culture, 2023
- Convolutional Neural Network Approach for Multimodal Biometric Recognition System for Banking Sector on Fusion of Face and Finger2023
- Spectrogram-based classification on vehicles with modified loud exhausts via convolutional neural networksApplied Acoustics, 2023
- An analysis of machine learning models for sentiment analysis of Tamil code-mixed dataComputer Speech & Language, 2022
- Benchmarking performance of machine and deep learning-based methodologies for Urdu text document classificationNeural Computing and Applications, 2020
- Audio classification using braided convolutional neural networksIET Signal Processing, 2020
- Literature review as a research methodology: An overview and guidelinesJournal of Business Research, 2019
- A review of affective computing: From unimodal analysis to multimodal fusionInformation Fusion, 2017