Siamese Interaction and Fine-Tuning Representation of Chinese Semantic Matching Algorithm Based on RoBERTa-wwm-ext
Open Access
- 15 April 2023
- journal article
- research article
- Published by Wiley in Mobile Information Systems
- Vol. 2023, 1-9
Abstract
Semantic matching research is the cornerstone of research in the fields of natural language similarity measurement and sensor ontology matching (OM). In the existing Chinese semantic matching methods, there are some shortcomings, such as the single dimension of semantic expression, the insufficient expression of context semantic relations, and the insufficient interaction of semantic information between different sentences. This paper proposes a Chinese semantic matching algorithm based on RoBERTa-wwm-ext with Siamese interaction and fine-tuning representation (RSIFR). The RSIFR model initializes the model with RoBERTa-wwm-ext as a vector of text. Firstly, a Siamese structure with embedded soft alignment attention mechanism and BiLSTM is constructed to realize the information interaction between two sentences. Secondly, LSTM-BiLSTM network structure is constructed to enhance the expression of semantic logic before and after sentences. Then, build a training model with fine-tuning mechanism. Fine tune the text’s eigenvector parameters through label supervision. Finally, the fusion vectors of the sentence pairs are inserted into the MLP network layer, resulting in semantic matching results. RSIFR model starts from a variety of dimensions, strengthens the expression ability of vectors to text semantic relations, deeply mines the semantic similarities and differences between different sentences, and generally improves the Chinese semantic matching performance. Experiments on the public dataset LCQMC show that our model outperforms existing Chinese semantic matching models.Keywords
Funding Information
- Natural Science Foundation of Guangxi Province (2019GXNSFDA185006, SKX212010053, SXX22138X002, 20190211-17, 20210104-1, 2022YCXS061)
- China Electronics Technology Group Corporation (2019GXNSFDA185006, SKX212010053, SXX22138X002, 20190211-17, 20210104-1, 2022YCXS061)
- Development Fund Project of Hebei Key Laboratory of Intelligent Information Perception and Processing (2019GXNSFDA185006, SKX212010053, SXX22138X002, 20190211-17, 20210104-1, 2022YCXS061)
- Guilin Science and Technology Development Program (2019GXNSFDA185006, SKX212010053, SXX22138X002, 20190211-17, 20210104-1, 2022YCXS061)
- Innovation Project of GUET Graduate Education (2019GXNSFDA185006, SKX212010053, SXX22138X002, 20190211-17, 20210104-1, 2022YCXS061)
This publication has 7 references indexed in Scilit:
- A hybrid approach of Weighted Fine-Tuned BERT extraction with deep Siamese Bi – LSTM model for semantic text similarity identificationMultimedia Tools and Applications, 2022
- Pre-Training With Whole Word Masking for Chinese BERTIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Frame-based Multi-level Semantics Representation for text matchingKnowledge-Based Systems, 2021
- Matching Sensor Ontologies With Multi-Context Similarity Measure and Parallel Compact Differential Evolution AlgorithmIEEE Sensors Journal, 2021
- Semantic Similarity Computing Model Based on Multi Model Fine-Grained Nonlinear FusionIEEE Access, 2021
- Cross-lingual text similarity exploiting neural machine translation modelsJournal of Information Science, 2020
- Siamese Recurrent Architectures for Learning Sentence SimilarityProceedings of the AAAI Conference on Artificial Intelligence, 2016