Key n -Gram Extractions and Analyses of Different Registers Based on Attention Network
Open Access
- 17 May 2021
- journal article
- research article
- Published by Wiley in Journal of Applied Mathematics
- Vol. 2021, 1-16
Abstract
Key-gram extraction can be seen as extracting -grams which can distinguish different registers. Keyword (as , 1-gram is the keyword) extraction models are generally carried out from two aspects, the feature extraction and the model design. By summarizing the advantages and disadvantages of existing models, we propose a novel key -gram extraction model “attentive -gram network” (ANN) based on the attention mechanism and multilayer perceptron, in which the attention mechanism scores each -gram in a sentence by mining the internal semantic relationship between words, and their importance is given by the scores. Experimental results on the real corpus show that the key -gram extracted from our model can distinguish a novel, news, and text book very well; the accuracy of our model is significantly higher than the baseline model. Also, we conduct experiments on key -grams extracted from these registers, which turned out to be well clustered. Furthermore, we make some statistical analyses of the results of key -gram extraction. We find that the key -grams extracted by our model are very explanatory in linguistics.Keywords
Funding Information
- Tsinghua University Humanities and Social Sciences Revitalization Project (2019THZWJC38, 20202000291, 62073284, 61603331, 2019YFB1406300)
- Project of Baidu Netcom Technology Co. Ltd. Open Source Course and Case Construction Based on the Deep Learning Framework PaddlePaddle (2019THZWJC38, 20202000291, 62073284, 61603331, 2019YFB1406300)
- Distributed Secure Estimation of Multi-sensor Systems Subject to Stealthy Attacks (2019THZWJC38, 20202000291, 62073284, 61603331, 2019YFB1406300)
- Multi-sensor-based Estimation Theory and Algorithms with Clustering Hierarchical Structures (2019THZWJC38, 20202000291, 62073284, 61603331, 2019YFB1406300)
- National Key Research and Development Program of China (2019THZWJC38, 20202000291, 62073284, 61603331, 2019YFB1406300)
This publication has 11 references indexed in Scilit:
- Keywords extraction in Chinese-Vietnamese bilingual news based on hypergraphInternational Journal of Distributed Sensor Networks, 2018
- A graph based keyword extraction model using collective node weightExpert Systems with Applications, 2018
- Unsupervised Key-phrase Extraction using Noun PhrasesInternational Journal of Computer Applications, 2017
- Deep learningNature, 2015
- 9. Linguistic manifestations of the space-time (dis)analogy2012
- Using lexical chains for keyword extractionInformation Processing & Management, 2007
- A Tutorial on the Cross-Entropy MethodAnnals of Operations Research, 2005
- Improved automatic keyword extraction given more linguistic knowledge2003
- Multilayer feedforward networks with a nonpolynomial activation function can approximate any functionNeural Networks, 1993
- The Reliability of Type-Token Ratios for the Oral Language of School Age ChildrenJournal of Speech, Language, and Hearing Research, 1989