Ancient Text Translation Model Optimized with GujiBERT and Entropy-SkipBERT
Open Access
- 15 November 2024
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 13 (22) , 4492
Abstract
To cope with the challenges posed by the complex linguistic structure and lexical polysemy in ancient texts, this study proposes a two-stage translation model. First, we combine GujiBERT, GCN, and LSTM to categorize ancient texts into historical and non-historical categories. This categorization lays the foundation for the subsequent translation task. To improve the efficiency of word vector generation and reduce the limitations of the traditional Word2Vec model, we integrated the entropy weight method in the hopping lattice training process and spliced the word vectors with GujiBERT. This improved method improves the efficiency of word vector generation and enhances the model’s ability to accurately represent lexical polysemy and grammatical structure in ancient documents through dependency weighting. In training the translation model, we used a different dataset for each text category, significantly improving the translation accuracy. Experimental results show that our categorization model improves the accuracy by 5% compared to GujiBERT. In contrast, the Entropy-SkipBERT improves the BLEU scores by 0.7 and 0.4 on historical and non-historical datasets. Ultimately, the proposed two-stage model improves the BLEU scores by 2.7 over the baseline model.Keywords
Funding Information
- Reform Project of Machine Translation Scientific Application of English for Cybersecurity Master’s Degree at North China University of Science and Technology (221006043084307)
This publication has 17 references indexed in Scilit:
- Enhancing Text Classification by Graph Neural Networks With Multi-Granular Topic-Aware GraphIEEE Access, 2023
- SA-SGRU: Combining Improved Self-Attention and Skip-GRU for Text ClassificationApplied Sciences, 2023
- Attention mechanism and skip-gram embedded phrasesComparative Legilinguistics, 2023
- Neural Machine Translation by Fusing Key Information of TextComputers, Materials & Continua, 2023
- An ensemble model for idioms and literal text classification using knowledge-enabled BERT in deep learningMeasurement: Sensors, 2022
- DPTCN: A novel deep CNN model for short text classificationJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology, 2021
- Pre-Training With Whole Word Masking for Chinese BERTIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Financial causal sentence recognition based on BERT-CNN text classificationThe Journal of Supercomputing, 2021
- Research on Short Text Classification Based on TextCNNJournal of Physics: Conference Series, 2021
- Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention MechanismApplied Sciences, 2020