Dina Amirova
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Dina Amirova has a total of 22 co-authors. Here are the top co-authors by number of shared publications:
- Aidana Karibayeva
Al-Farabi Kazakh National University
8 shared publications - Balzhan Abduali
Al-Farabi Kazakh National University
7 shared publications - Diana Rakhimova
Al-Farabi Kazakh National University
6 shared publications - Ualsher Tukeyev
Al-Farabi Kazakh National University
6 shared publications - Vladislav Karyukin
Al-Farabi Kazakh National University
6 shared publications - Rashid Aliyev
Faculty of Information Technology and Artificial Intelligence, Farabi University, Almaty 050040, Kazakhstan
4 shared publications
Latest Publications
- Open Access
Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results
Speech-to-speech translation (S2ST) among Turkic languages is rarely studied due to the scarcity of parallel speech corpora and the complex morphology. In this work, we develop low-resource S2ST systems translating from Kazakh to Turkish, Tatar, and Uzbek, and examine which architectural choices proved ineffective and why. Three architectures were trained on a synthetic parallel speech corpus: (i) a scratch-trained, morphology-aware Conformer encoder with a morpheme-level connectionist temporal classification (CTC) loss, a vector-quantized (VQ) codebook, and a semantic anchor; (ii) a pretrained Whisper encoder with a text decoder conditioned on the target language by feature-wise linear modulation (FiLM); and (iii) a text-mediated two-pass model that predicts Turkic speech units from the decoder states and converts them to speech with a Turkic unit vocoder. Text output was scored with BLEU and speech output with ASR-BLEU, that is, BLEU on automatic transcripts. The scratch-trained model and about ten variants scored ASR-BLEU 0. The Whisper-based speech-to-text model reached 19.49 BLEU on synthetic test speech and 3.56 on natural speech. The final speech-to-speech model reached 3.25 ASR-BLEU on about 1500 synthetic test utterances per language, 5.1 times the English-unit baseline, and 0.86 ASR-BLEU on natural speech. These scores show feasibility, not usable quality.
Open AccessSpeech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results
Speech-to-speech translation (S2ST) among Turkic languages is rarely studied due to the scarcity of parallel speech corpora and the complex morphology. In this work, we develop low-resource S2ST systems translating from Kazakh to Turkish, Tatar, and Uzbek, and examine which architectural choices proved ineffective and why. Three architectures were trained on a synthetic parallel speech corpus: (i) a scratch-trained, morphology-aware Conformer encoder with a morpheme-level connectionist temporal classification (CTC) loss, a vector-quantized (VQ) codebook, and a semantic anchor; (ii) a pretrained Whisper encoder with a text decoder conditioned on the target language by feature-wise linear modulation (FiLM); and (iii) a text-mediated two-pass model that predicts Turkic speech units from the decoder states and converts them to speech with a Turkic unit vocoder. Text output was scored with BLEU and speech output with ASR-BLEU, that is, BLEU on automatic transcripts. The scratch-trained model and about ten variants scored ASR-BLEU 0. The Whisper-based speech-to-text model reached 19.49 BLEU on synthetic test speech and 3.56 on natural speech. The final speech-to-speech model reached 3.25 ASR-BLEU on about 1500 synthetic test utterances per language, 5.1 times the English-unit baseline, and 0.86 ASR-BLEU on natural speech. These scores show feasibility, not usable quality.
- Open Access
Toward an Adaptive Speech-to-Summary Pipeline for Turkic Languages: Language-Specific ASR, Conditional Morphology-Aware Correction, Pivot Translation, and Summarization
Speech-to-Summary cascade performance for low-resource, morphologically rich languages depends on interactions among language, recognition errors, and translation direction. This study develops an adaptive Speech-to-Summary pipeline for six Turkic languages—Azerbaijani, Kazakh, Kyrgyz, Turkish, Turkmen, and Uzbek—integrating ASR, conditional morphology-aware post-processing (MAPP), MT, pivot routing, and summarization. Whisper-family models lead external benchmarks for Turkish, Kazakh, and Azerbaijani; MMS for Kyrgyz, Uzbek, and Turkmen. The Uzbek benchmark-best model proves unreliable on real recordings, requiring a different deployed model. Pause-based segmentation reduces ASR errors for Kazakh, Azerbaijani, Uzbek, and Kyrgyz but is not universally beneficial. MAPP shows no consistent improvement on strong real-ASR outputs; only 3.6% of real Uzbek substitution errors are reachable by the correction rules, supporting conditional rather than mandatory correction. MT evaluation across 22 FLORES-200 directions reveals direction-dependent performance and sensitivity to script compatibility. Pivot selection is language-dependent: direct-Kazakh routing performs better for Kazakh, Turkmen, and Turkish, whereas Azerbaijani benefits from an English pivot. The results support replacing a fixed cascade with a configurable architecture selecting ASR, segmentation, correction, MT, and pivot routing by language, error patterns, and deployment constraints. Routing decisions are currently made offline based on empirical evidence; automatic routing is left for future work.
Open AccessToward an Adaptive Speech-to-Summary Pipeline for Turkic Languages: Language-Specific ASR, Conditional Morphology-Aware Correction, Pivot Translation, and Summarization
Speech-to-Summary cascade performance for low-resource, morphologically rich languages depends on interactions among language, recognition errors, and translation direction. This study develops an adaptive Speech-to-Summary pipeline for six Turkic languages—Azerbaijani, Kazakh, Kyrgyz, Turkish, Turkmen, and Uzbek—integrating ASR, conditional morphology-aware post-processing (MAPP), MT, pivot routing, and summarization. Whisper-family models lead external benchmarks for Turkish, Kazakh, and Azerbaijani; MMS for Kyrgyz, Uzbek, and Turkmen. The Uzbek benchmark-best model proves unreliable on real recordings, requiring a different deployed model. Pause-based segmentation reduces ASR errors for Kazakh, Azerbaijani, Uzbek, and Kyrgyz but is not universally beneficial. MAPP shows no consistent improvement on strong real-ASR outputs; only 3.6% of real Uzbek substitution errors are reachable by the correction rules, supporting conditional rather than mandatory correction. MT evaluation across 22 FLORES-200 directions reveals direction-dependent performance and sensitivity to script compatibility. Pivot selection is language-dependent: direct-Kazakh routing performs better for Kazakh, Turkmen, and Turkish, whereas Azerbaijani benefits from an English pivot. The results support replacing a fixed cascade with a configurable architecture selecting ASR, segmentation, correction, MT, and pivot routing by language, error patterns, and deployment constraints. Routing decisions are currently made offline based on empirical evidence; automatic routing is left for future work.
- 18 September 2026
- Published by Springer Nature
- p. 285-298
From Speech to Readable Text: Automatic Punctuation in ASR Systems of the Kazakh Language
- 18 September 2026
- Published by Springer Nature
- p. 285-298
From Speech to Readable Text: Automatic Punctuation in ASR Systems of the Kazakh Language
- Open Access
- 4 August 2026
- Published by MDPI AG in Big Data and Cognitive Computing
- Vol. 10 (8) , 259
Adapting Large-Scale Foundation Models for Turkic Speech-to-Speech Translation: Fine-Tuned Cascade and Direct Approaches
This paper presents a novel approach to speech-to-speech (STS) translation for low-resource Turkic languages. Today, STS has progressed rapidly for high-resource languages; the Turkic family remains significantly underrepresented. Consequently, it is quite challenging to develop a reliable speech translation system for these languages. To address this issue, we have developed two speech translation systems (STS) specifically tailored to Turkic languages with limited resources. The first, TurkicCascadeSTS, is a cascaded system that combines a fine-tuned Whisper-medium speech recognition model, GPT translation, and separate speech synthesis models for each language. The second system is a direct speech translation model based on a fine-tuned SeamlessM4Tv2. Both systems have been tested for translation into Turkic languages, using 24,656 audio recordings per language. The TurkicCascadeSTS system delivered far better results: the average BLEU score rose from 4.68 to 30.60; the METEOR score rose from 15.03 to 44.42; and the word error rate (WER) also fell significantly. These improvements are due to the fact that each module of the system was individually fine-tuned to account for the specific characteristics of each language. Although SeamlessM4Tv2 sometimes produces clearer audio, TurkicCascadeSTS generally delivers higher speech and translation quality for all language pairs. This demonstrates that modular, specially tuned systems are an effective solution for translation into Turkic languages, particularly given their complex structure and limited linguistic resources. Such systems could benefit more than 200 million native speakers of Turkic languages.
- 4 August 2026
- Published by MDPI AG in Big Data and Cognitive Computing
- Vol. 10 (8) , 259
Open AccessAdapting Large-Scale Foundation Models for Turkic Speech-to-Speech Translation: Fine-Tuned Cascade and Direct Approaches
This paper presents a novel approach to speech-to-speech (STS) translation for low-resource Turkic languages. Today, STS has progressed rapidly for high-resource languages; the Turkic family remains significantly underrepresented. Consequently, it is quite challenging to develop a reliable speech translation system for these languages. To address this issue, we have developed two speech translation systems (STS) specifically tailored to Turkic languages with limited resources. The first, TurkicCascadeSTS, is a cascaded system that combines a fine-tuned Whisper-medium speech recognition model, GPT translation, and separate speech synthesis models for each language. The second system is a direct speech translation model based on a fine-tuned SeamlessM4Tv2. Both systems have been tested for translation into Turkic languages, using 24,656 audio recordings per language. The TurkicCascadeSTS system delivered far better results: the average BLEU score rose from 4.68 to 30.60; the METEOR score rose from 15.03 to 44.42; and the word error rate (WER) also fell significantly. These improvements are due to the fact that each module of the system was individually fine-tuned to account for the specific characteristics of each language. Although SeamlessM4Tv2 sometimes produces clearer audio, TurkicCascadeSTS generally delivers higher speech and translation quality for all language pairs. This demonstrates that modular, specially tuned systems are an effective solution for translation into Turkic languages, particularly given their complex structure and limited linguistic resources. Such systems could benefit more than 200 million native speakers of Turkic languages.
- Open Access
- 30 June 2026
- Published by MDPI AG in Information
- Vol. 17 (7) , 639
Improving Kazakh Abstractive Summarization with LLMs via Domain Adaptation and Self-Correction
Recent advances in large language models (LLMs) have improved abstractive text summarization, yet their benefits remain limited for low-resource and morphologically rich languages such as Kazakh. This study investigates the adaptation of the instruction-tuned Gemma-3-4B-it (Google LLC, Mountain View, CA, USA) model for Kazakh news summarization. We propose a two-stage training pipeline combining domain-adaptive pretraining (DAPT) on an unlabeled Kazakh news corpus with parameter-efficient supervised fine-tuning (SFT) using LoRA on summarization datasets. We also examine an inference-time self-correction step in which the model revises its initial summaries to improve consistency and reduce factual errors. Experiments are conducted on Kazakh news data, including a translated version of XSum and corpora collected from BAQ and TengriNews. We compare the baseline, SFT-only, DAPT-only, and DAPT + SFT configurations, with and without self-correction. Evaluation is performed using ROUGE, BERTScore, chrF++, human assessment, and qualitative linguistic analysis. The results show that DAPT combined with SFT achieves the strongest overall performance, while self-correction further improves automatic scores in the DAPT + SFT setting. However, human evaluation indicates that metric gains do not always correspond to higher factual faithfulness or informativeness. These findings highlight the need for domain adaptation and careful evaluation when applying LLMs to Kazakh summarization.
- 30 June 2026
- Published by MDPI AG in Information
- Vol. 17 (7) , 639
Open AccessImproving Kazakh Abstractive Summarization with LLMs via Domain Adaptation and Self-Correction
Recent advances in large language models (LLMs) have improved abstractive text summarization, yet their benefits remain limited for low-resource and morphologically rich languages such as Kazakh. This study investigates the adaptation of the instruction-tuned Gemma-3-4B-it (Google LLC, Mountain View, CA, USA) model for Kazakh news summarization. We propose a two-stage training pipeline combining domain-adaptive pretraining (DAPT) on an unlabeled Kazakh news corpus with parameter-efficient supervised fine-tuning (SFT) using LoRA on summarization datasets. We also examine an inference-time self-correction step in which the model revises its initial summaries to improve consistency and reduce factual errors. Experiments are conducted on Kazakh news data, including a translated version of XSum and corpora collected from BAQ and TengriNews. We compare the baseline, SFT-only, DAPT-only, and DAPT + SFT configurations, with and without self-correction. Evaluation is performed using ROUGE, BERTScore, chrF++, human assessment, and qualitative linguistic analysis. The results show that DAPT combined with SFT achieves the strongest overall performance, while self-correction further improves automatic scores in the DAPT + SFT setting. However, human evaluation indicates that metric gains do not always correspond to higher factual faithfulness or informativeness. These findings highlight the need for domain adaptation and careful evaluation when applying LLMs to Kazakh summarization.
- Open Access
- 13 June 2026
- Published by MDPI AG in Information
- Vol. 17 (6) , 594
CSE-Guided Linguistically Constrained Morphological Segmentation for Turkmen
This paper presents a linguistically constrained neural approach to morphological segmentation for low-resource Turkic languages, with a case study on Turkmen. The proposed method combines large-scale training data generated by a Complete Set of Endings (CSE) model with a neural architecture augmented with explicit phonological inductive biases. Unlike prior FEMSeg-based architectures that rely on convolutional and Transformer layers for implicit feature learning, the proposed model, LCMSeg (Linguistically Constrained Morphological Segmentation), introduces vowel/consonant indicators and harmony-class embeddings, both of which are directly derived from linguistic rules. The constraints are implemented as inductive biases. The CSE framework serves as a data-generation mechanism, producing a segmented corpus of 270k sentences used for training. The neural model learns to approximate the segmentation function induced by the CSE annotations while generalizing beyond the limitations of rule-based methods. Experiments conducted on training sets of 10k to 80k sentences demonstrate consistent improvements, achieving up to 99.76% token accuracy and 99.53% morpheme accuracy. Evaluation on the FLORES-200 benchmark confirms strong generalization under domain shift, with harmony consistency reaching 98.9%. The results show that explicitly encoding phonological structure provides a strong inductive bias, particularly beneficial in low-resource settings. The proposed framework offers a scalable and linguistically grounded solution for morphological segmentation in Turkic languages.
- 13 June 2026
- Published by MDPI AG in Information
- Vol. 17 (6) , 594
Open AccessCSE-Guided Linguistically Constrained Morphological Segmentation for Turkmen
This paper presents a linguistically constrained neural approach to morphological segmentation for low-resource Turkic languages, with a case study on Turkmen. The proposed method combines large-scale training data generated by a Complete Set of Endings (CSE) model with a neural architecture augmented with explicit phonological inductive biases. Unlike prior FEMSeg-based architectures that rely on convolutional and Transformer layers for implicit feature learning, the proposed model, LCMSeg (Linguistically Constrained Morphological Segmentation), introduces vowel/consonant indicators and harmony-class embeddings, both of which are directly derived from linguistic rules. The constraints are implemented as inductive biases. The CSE framework serves as a data-generation mechanism, producing a segmented corpus of 270k sentences used for training. The neural model learns to approximate the segmentation function induced by the CSE annotations while generalizing beyond the limitations of rule-based methods. Experiments conducted on training sets of 10k to 80k sentences demonstrate consistent improvements, achieving up to 99.76% token accuracy and 99.53% morpheme accuracy. Evaluation on the FLORES-200 benchmark confirms strong generalization under domain shift, with harmony consistency reaching 98.9%. The results show that explicitly encoding phonological structure provides a strong inductive bias, particularly beneficial in low-resource settings. The proposed framework offers a scalable and linguistically grounded solution for morphological segmentation in Turkic languages.
- Open Access
A Lightweight Multimodal Architecture for Punctuation Restoration in Kazakh ASR
In this paper, we first present a multimodal architecture called CrossAttn-v1. This model is designed to recover punctuation marks in Kazakh and combines contextual XLM-RoBERTa-large text embeddings with the Whisper large-v3 encoder states via a cross-attention mechanism. In addition, a 4-dimensional prosodic vector and a CRF output layer are used. The model was trained using an adapted Whisper ASR model on 33,332 utterances from the KazakhTTS2 corpus. After adaptation, the word error rate decreased from 45.7% to 4.25%. On the in-domain test set (56,396 tokens), CrossAttn-v1 achieved F1-macro = 0.8485 for recovering five-class punctuation marks. Furthermore, CrossAttn-v1 outperformed the GPT-4o zero-shot model by +0.294 F1 and the M3 Hybrid model based on prosody alone by +0.070 F1. The class analysis showed that the Whisper encoder states were particularly useful for prosody-dependent punctuation. For example, it outperformed M3 Hybrid by +9.5 percentage points on the QUESTION mark and by +20.2 percentage points on the EXCLAIM mark. On 883 out-of-domain natural speech recordings, the model performed similarly to the text-only baseline model (Δ = −0.041, not significant), suggesting that domain mismatch in the Whisper training corpus was a major factor limiting generalization.
Open AccessA Lightweight Multimodal Architecture for Punctuation Restoration in Kazakh ASR
In this paper, we first present a multimodal architecture called CrossAttn-v1. This model is designed to recover punctuation marks in Kazakh and combines contextual XLM-RoBERTa-large text embeddings with the Whisper large-v3 encoder states via a cross-attention mechanism. In addition, a 4-dimensional prosodic vector and a CRF output layer are used. The model was trained using an adapted Whisper ASR model on 33,332 utterances from the KazakhTTS2 corpus. After adaptation, the word error rate decreased from 45.7% to 4.25%. On the in-domain test set (56,396 tokens), CrossAttn-v1 achieved F1-macro = 0.8485 for recovering five-class punctuation marks. Furthermore, CrossAttn-v1 outperformed the GPT-4o zero-shot model by +0.294 F1 and the M3 Hybrid model based on prosody alone by +0.070 F1. The class analysis showed that the Whisper encoder states were particularly useful for prosody-dependent punctuation. For example, it outperformed M3 Hybrid by +9.5 percentage points on the QUESTION mark and by +20.2 percentage points on the EXCLAIM mark. On 883 out-of-domain natural speech recordings, the model performed similarly to the text-only baseline model (Δ = −0.041, not significant), suggesting that domain mismatch in the Whisper training corpus was a major factor limiting generalization.
- Top Cited PapersOpen Access
An Integrated Approach to Adapting Open-Source AI Models for Machine Translation of Low-Resource Turkic Languages
This study presents the application of free, open-source artificial intelligence (AI) techniques to advance machine translation for low-resource Turkic languages such as Kazakh, Azerbaijani, Kyrgyz, Turkish, Turkmen, and Uzbek. This machine translation problem for Turkic languages is part of a project to generate meeting minutes from speech transcripts. Due to limited parallel corpora and underdeveloped linguistic tools for these languages, traditional machine translation approaches often underperform. The goal is to reduce digital inequality for these languages and to support scalability. We investigate the effectiveness of free open-source pre-trained specialized and general-purpose AI models for morphologically rich state Turkic languages. This research includes developing parallel corpora for six Turkic languages, fine-tuning, and performance evaluation using BLEU, WER, TER, and chrF metrics. The parallel corpora for five pair languages, each of 300,000 and 500,000 sentences, were generated and cleaned. The results for corpora 500,000 parallel sentences show significant improvements compared with baseline NLLB-200 1.3B on average: BLEU increased by 23.81 points, chrF increased by 26.05 points, and WER and TER decreased by 0.36 and 33.95, respectively, after cleaning and fine-tuning. Six Turkic-language multilingual parallel corpora of 3 885 542 sentences were developed and the fine-tuning of NLLB-200 1.3B shows the following, compared with the results for 500,000 cleaned corpus: BLEU increased by 4.3 points, chrF increased by 1.7 points, and WER and TER decreased by 0.1 and 4.75, respectively These results demonstrate the high efficiency of corpus cleaning and synthetic data generation to improve the quality of machine translation for low-resource Turkic languages using AI models. These results were confirmed by external evaluation on the FLORES 200 dataset and human evaluation. The scientific contribution of this article is the development of a methodology for generating parallel corpora using a specialized AI model of machine translation and fine-tuning the specialized AI model on the created corpora, creating new multilingual parallel corpora of Azerbaijan–Kazakh, Kyrgyz–Kazakh, Turkish–Kazakh, Turkmen–Kazakh, and Uzbek–Kazakh pairs using the proposed methodology, cleaning them, and conducting fine-tuning experiments.
Top Cited PapersOpen AccessAn Integrated Approach to Adapting Open-Source AI Models for Machine Translation of Low-Resource Turkic Languages
This study presents the application of free, open-source artificial intelligence (AI) techniques to advance machine translation for low-resource Turkic languages such as Kazakh, Azerbaijani, Kyrgyz, Turkish, Turkmen, and Uzbek. This machine translation problem for Turkic languages is part of a project to generate meeting minutes from speech transcripts. Due to limited parallel corpora and underdeveloped linguistic tools for these languages, traditional machine translation approaches often underperform. The goal is to reduce digital inequality for these languages and to support scalability. We investigate the effectiveness of free open-source pre-trained specialized and general-purpose AI models for morphologically rich state Turkic languages. This research includes developing parallel corpora for six Turkic languages, fine-tuning, and performance evaluation using BLEU, WER, TER, and chrF metrics. The parallel corpora for five pair languages, each of 300,000 and 500,000 sentences, were generated and cleaned. The results for corpora 500,000 parallel sentences show significant improvements compared with baseline NLLB-200 1.3B on average: BLEU increased by 23.81 points, chrF increased by 26.05 points, and WER and TER decreased by 0.36 and 33.95, respectively, after cleaning and fine-tuning. Six Turkic-language multilingual parallel corpora of 3 885 542 sentences were developed and the fine-tuning of NLLB-200 1.3B shows the following, compared with the results for 500,000 cleaned corpus: BLEU increased by 4.3 points, chrF increased by 1.7 points, and WER and TER decreased by 0.1 and 4.75, respectively These results demonstrate the high efficiency of corpus cleaning and synthetic data generation to improve the quality of machine translation for low-resource Turkic languages using AI models. These results were confirmed by external evaluation on the FLORES 200 dataset and human evaluation. The scientific contribution of this article is the development of a methodology for generating parallel corpora using a specialized AI model of machine translation and fine-tuning the specialized AI model on the created corpora, creating new multilingual parallel corpora of Azerbaijan–Kazakh, Kyrgyz–Kazakh, Turkish–Kazakh, Turkmen–Kazakh, and Uzbek–Kazakh pairs using the proposed methodology, cleaning them, and conducting fine-tuning experiments.
- Open Access
- 5 December 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (24) , 12880
The Development and Experimental Evaluation of a Multilingual Speech Corpus for Low-Resource Turkic Languages
The development of parallel audio corpora for Turkic languages, such as Kazakh, Uzbek, and Tatar, remains a significant challenge in the development of multilingual speech synthesis, recognition systems, and machine translation. These languages are low-resource in speech technologies, lacking sufficiently large audio datasets with aligned transcriptions that are crucial for modern recognition, synthesis, and understanding systems. This article presents the development and experimental evaluation of a speech corpus focused on Turkic languages, intended for use in speech synthesis and automatic translation tasks. The primary objective is to create parallel audio corpora using a cascade generation method, which combines artificial intelligence and text-to-speech (TTS) technologies to generate both audio and text, and to evaluate the quality and suitability of the generated data. To evaluate the quality of synthesized speech, metrics measuring naturalness, intonation, expressiveness, and linguistic adequacy were applied. As a result, a multimodal (Kazakh–Turkish, Kazakh–Tatar, Kazakh–Uzbek) corpus was created, combining high-quality natural Kazakh audio with transcription and translation, along with synthetic audio in Turkish, Tatar, and Uzbek. These corpora offer a unique resource for speech and text processing research, enabling the integration of ASR, MT, TTS, and speech-to-speech translation (STS).
- 5 December 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (24) , 12880
Open AccessThe Development and Experimental Evaluation of a Multilingual Speech Corpus for Low-Resource Turkic Languages
The development of parallel audio corpora for Turkic languages, such as Kazakh, Uzbek, and Tatar, remains a significant challenge in the development of multilingual speech synthesis, recognition systems, and machine translation. These languages are low-resource in speech technologies, lacking sufficiently large audio datasets with aligned transcriptions that are crucial for modern recognition, synthesis, and understanding systems. This article presents the development and experimental evaluation of a speech corpus focused on Turkic languages, intended for use in speech synthesis and automatic translation tasks. The primary objective is to create parallel audio corpora using a cascade generation method, which combines artificial intelligence and text-to-speech (TTS) technologies to generate both audio and text, and to evaluate the quality and suitability of the generated data. To evaluate the quality of synthesized speech, metrics measuring naturalness, intonation, expressiveness, and linguistic adequacy were applied. As a result, a multimodal (Kazakh–Turkish, Kazakh–Tatar, Kazakh–Uzbek) corpus was created, combining high-quality natural Kazakh audio with transcription and translation, along with synthetic audio in Turkish, Tatar, and Uzbek. These corpora offer a unique resource for speech and text processing research, enabling the integration of ASR, MT, TTS, and speech-to-speech translation (STS).
- Open Access
- 10 October 2025
- Published by MDPI AG in Information
- Vol. 16 (10) , 879
Speech Recognition and Synthesis Models and Platforms for the Kazakh Language
With the rapid development of artificial intelligence and machine learning technologies, automatic speech recognition (ASR) and text-to-speech (TTS) have become key components of the digital transformation of society. The Kazakh language, as a representative of the Turkic language family, remains a low-resource language with limited audio corpora, language models, and high-quality speech synthesis systems. This study provides a comprehensive analysis of existing speech recognition and synthesis models, emphasizing their applicability and adaptation to the Kazakh language. Special attention is given to linguistic and technical barriers, including the agglutinative structure, rich vowel system, and phonemic variability. Both open-source and commercial solutions were evaluated, including Whisper, GPT-4 Transcribe, ElevenLabs, OpenAI TTS, Voiser, KazakhTTS2, and TurkicTTS. Speech recognition systems were assessed using BLEU, WER, TER, chrF, and COMET, while speech synthesis was evaluated with MCD, PESQ, STOI, and DNSMOS, thus covering both lexical–semantic and acoustic–perceptual characteristics. The results demonstrate that, for speech-to-text (STT), the strongest performance was achieved by Soyle on domain-specific data (BLEU 74.93, WER 18.61), while Voiser showed balanced accuracy (WER 40.65–37.11, chrF 80.88–84.51) and GPT-4 Transcribe achieved robust semantic preservation (COMET up to 1.02). In contrast, Whisper performed weakest (WER 77.10, BLEU 13.22), requiring further adaptation for Kazakh. For text-to-speech (TTS), KazakhTTS2 delivered the most natural perceptual quality (DNSMOS 8.79–8.96), while OpenAI TTS achieved the best spectral accuracy (MCD 123.44–117.11, PESQ 1.14). TurkicTTS offered reliable intelligibility (STOI 0.15, PESQ 1.16), and ElevenLabs produced natural but less spectrally accurate speech.
- 10 October 2025
- Published by MDPI AG in Information
- Vol. 16 (10) , 879
Open AccessSpeech Recognition and Synthesis Models and Platforms for the Kazakh Language
With the rapid development of artificial intelligence and machine learning technologies, automatic speech recognition (ASR) and text-to-speech (TTS) have become key components of the digital transformation of society. The Kazakh language, as a representative of the Turkic language family, remains a low-resource language with limited audio corpora, language models, and high-quality speech synthesis systems. This study provides a comprehensive analysis of existing speech recognition and synthesis models, emphasizing their applicability and adaptation to the Kazakh language. Special attention is given to linguistic and technical barriers, including the agglutinative structure, rich vowel system, and phonemic variability. Both open-source and commercial solutions were evaluated, including Whisper, GPT-4 Transcribe, ElevenLabs, OpenAI TTS, Voiser, KazakhTTS2, and TurkicTTS. Speech recognition systems were assessed using BLEU, WER, TER, chrF, and COMET, while speech synthesis was evaluated with MCD, PESQ, STOI, and DNSMOS, thus covering both lexical–semantic and acoustic–perceptual characteristics. The results demonstrate that, for speech-to-text (STT), the strongest performance was achieved by Soyle on domain-specific data (BLEU 74.93, WER 18.61), while Voiser showed balanced accuracy (WER 40.65–37.11, chrF 80.88–84.51) and GPT-4 Transcribe achieved robust semantic preservation (COMET up to 1.02). In contrast, Whisper performed weakest (WER 77.10, BLEU 13.22), requiring further adaptation for Kazakh. For text-to-speech (TTS), KazakhTTS2 delivered the most natural perceptual quality (DNSMOS 8.79–8.96), while OpenAI TTS achieved the best spectral accuracy (MCD 123.44–117.11, PESQ 1.14). TurkicTTS offered reliable intelligibility (STOI 0.15, PESQ 1.16), and ElevenLabs produced natural but less spectrally accurate speech.