Sentence and Document Representation Learning
Open Access
- 24 August 2023
- book chapter
- Published by Springer Nature
- p. 81-125
Abstract
Sentence and document are high-level linguistic units of natural languages. Representation learning of sentences and documents remains a core and challenging task because many important applications of natural language processing (NLP) lie in understanding sentences and documents. This chapter first introduces symbolic methods to sentence and document representation learning. Then we extensively introduce neural network-based methods for the far-reaching language modeling task, including feed-forward neural networks, convolutional neural networks, recurrent neural networks, and Transformers. Regarding the characteristics of a document consisting of multiple sentences, we particularly introduce memory-based and hierarchical approaches to document representation learning. Finally, we present representative applications of sentence and document representation, including text classification, sequence labeling, reading comprehension, question answering, information retrieval, and sequence-to-sequence generation.Keywords
This publication has 49 references indexed in Scilit:
- Effective Use of Word Order for Text Categorization with Convolutional Neural Networks2015
- A Convolutional Neural Network for Modelling Sentences2014
- Convolutional Neural Networks for Sentence Classification2014
- HMM Based POS Tagger for Hindi2013
- Neural Network Language Model with Cache2012
- Scaling question answering to the webACM Transactions on Information Systems, 2001
- BLEU2001
- Long Short-Term MemoryNeural Computation, 1997
- Estimation of probabilities from sparse data for the language model component of a speech recognizerIEEE Transactions on Acoustics, Speech, and Signal Processing, 1987
- THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERSBiometrika, 1953