Fu Wang
Research Subjects
Fu Wang has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Chenhao Zhang
Guilin, China
1 shared publication - Xianwen Liao
School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China
1 shared publication - Yong Wang
School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China
1 shared publication - Yongzhong Huang
Guilin University of Electronic Technology
1 shared publication - Yongzhuang Wei
Guangxi Key Laboratory of Cryptography and Information Security, Guilin University of Electronic Technology, Guilin, China
1 shared publication
Latest Publications
- Open Access
- 29 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 3384-3399
Efficient Estimate of Sentence's Representation Based on the Difference Semantics Model
Sentence representation is an important research hotspot in natural language processing (NLP) since it can map the semantics of sentences into semantics vectors, thereby effectively solving complex semantics computing problems. Recently, sentence representations are mainly obtained by indirect means. Specifically, for sentence representations obtained by unsupervised means, they are often calculated by the weighted sum of embeddings of tokens in sentences; for sentence representations obtained by self-supervised or supervised means, they are often derived from intermediate encodings of sentences in prediction tasks. For example, Google's BERT and MUSE respectively use the embedding of [CLS] in the next sentence prediction task and intermediate encodings of sentences in the translation bridge task as sentence representations. In this paper, we use the observed semantics increment feature of sentences to directly model the semantics function of sentences. To be able to use the existing neural network language model to approximate the semantics function, we first implement the first-order Taylor expansion on the semantics function to obtain a difference semantics model and then add it to BERT as a subtask to perform self-supervised fine-tuning. Finally, we get a new sentence representation model S-BERT. S-BERT achieves the state-of-the-art performance on many datasets in Chinese, English, and Vietnamese.
- 29 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 3384-3399
Open AccessEfficient Estimate of Sentence's Representation Based on the Difference Semantics Model
Sentence representation is an important research hotspot in natural language processing (NLP) since it can map the semantics of sentences into semantics vectors, thereby effectively solving complex semantics computing problems. Recently, sentence representations are mainly obtained by indirect means. Specifically, for sentence representations obtained by unsupervised means, they are often calculated by the weighted sum of embeddings of tokens in sentences; for sentence representations obtained by self-supervised or supervised means, they are often derived from intermediate encodings of sentences in prediction tasks. For example, Google's BERT and MUSE respectively use the embedding of [CLS] in the next sentence prediction task and intermediate encodings of sentences in the translation bridge task as sentence representations. In this paper, we use the observed semantics increment feature of sentences to directly model the semantics function of sentences. To be able to use the existing neural network language model to approximate the semantics function, we first implement the first-order Taylor expansion on the semantics function to obtain a difference semantics model and then add it to BERT as a subtask to perform self-supervised fine-tuning. Finally, we get a new sentence representation model S-BERT. S-BERT achieves the state-of-the-art performance on many datasets in Chinese, English, and Vietnamese.