Duration Modeling with Semi-Markov Conditional Random Fields for Keyphrase Extraction
- 19 September 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 33 (4) , 1453-1466
Abstract
Existing methods for keyphrase extraction need preprocessing to generate candidate phrase or post-processing to transform keyword into keyphrase. In this paper, we propose a novel approach called duration modeling with semi-Markov Conditional Random Fields (DM-SMCRFs) for keyphrase extraction. First of all, based on the property of semi-Markov chain, DM-SMCRFs can encode segment-level features and sequentially classify the phrase in the sentence as keyphrase or non-keyphrase. Second, by assuming the independence between state transition and state duration, DM-SMCRFs model the distribution of duration (length) of keyphrases to further explore state duration information, which can help identify the size of keyphrase. Based on the convexity of parametric duration feature derived from duration distribution, a constrained Viterbi algorithm is derived to improve the performance of decoding in DM-SMCRFs. We thoroughly evaluate the performance of DM-SMCRFs on the datasets from various domains. The experimental results demonstrate the effectiveness of proposed model.Keywords
All Related Versions
Funding Information
- Mr. MW Lau of CityU (9220083)
This publication has 20 references indexed in Scilit:
- Mining Quality Phrases from Massive Text Corpora2015
- Automatic Keyphrase Extraction: A Survey of the State of the Art2014
- Introduction to Information RetrievalJournal of the American Society for Information Science and Technology, 2009
- Information extraction from research papers using conditional random fieldsInformation Processing & Management, 2006
- Improving the scalability of semi-Markov conditional random fields for named entity recognition2006
- KEA2004
- MAP speaker adaptation of state duration distributions for speech recognitionIEEE Transactions on Speech and Audio Processing, 2002
- Automatic Keyword Extraction Using Domain Knowledge2001
- Robust parametric modeling of durations in hidden Markov modelsIEEE Transactions on Speech and Audio Processing, 1996
- Term-weighting approaches in automatic text retrievalInformation Processing & Management, 1988