A Continuously Growing Dataset of Sentential Paraphrases
Authors:
Preprint
Abstract
A major challenge in paraphrase research is the lack of parallel corpora. In this paper, we present a new method to collect large-scale sentential paraphrases from Twitter by linking tweets through shared URLs. The main advantage of our method is its simplicity, as it gets rid of the classifier or human in the loop needed to select data before annotation and subsequent application of paraphrase identification algorithms in the previous work. We present the largest human-labeled paraphrase corpus to date of 51,524 sentence pairs and the first cross-domain benchmarking for automatic paraphrase identification. In addition, we show that more than 30,000 new sentential paraphrases can be easily and continuously captured every month at ~70% precision, and demonstrate their utility for downstream NLP tasks through phrasal paraphrase extraction. We make our code and data freely available.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Large-Scale Information Extraction from Textual Definitions through Deep Syntactic and Semantic AnalysisTransactions of the Association for Computational Linguistics, 2015
- Exploiting Parallel News Streams for Unsupervised Event ExtractionTransactions of the Association for Computational Linguistics, 2015
- From Paraphrase Database to Compositional Paraphrase Model and BackTransactions of the Association for Computational Linguistics, 2015
- Extracting Lexically Divergent Paraphrases from TwitterTransactions of the Association for Computational Linguistics, 2014
- Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual EvidenceTransactions of the Association for Computational Linguistics, 2014
- Paraphrase acquisition via crowdsourcing and machine learningACM Transactions on Intelligent Systems and Technology, 2013
- Generating Phrasal and Sentential Paraphrases: A Survey of Data-Driven MethodsComputational Linguistics, 2010
- Inter-Coder Agreement for Computational LinguisticsComputational Linguistics, 2008
- Constructing Corpora for the Development and Evaluation of Paraphrase SystemsComputational Linguistics, 2008
- THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1New Phytologist, 1912