Are n-gram Categories Helpful in Text Classification?
Authors:
- 15 June 2020
- book chapter
- Published by Springer Nature
- Vol. 12138, 524-537
Abstract
Character n-grams are widely used in text categorization problems and are the single most successful type of feature in authorship attribution. Their primary advantage is language independence, as they can be applied to a new language with no additional effort. Typed character n-grams reflect information about their content and context. According to previous research, typed character n-grams improve the accuracy of authorship attribution. This paper examines their effectiveness in three domains: authorship attribution, author profiling and sentiment analysis. The problem of a very high number of features is tackled with distributed Apache Spark processing.Keywords
This publication has 11 references indexed in Scilit:
- Enriching Word Vectors with Subword InformationTransactions of the Association for Computational Linguistics, 2017
- Distributed Classification of Text Documents on Apache Spark PlatformLecture Notes in Computer Science, 2016
- Not All Character N-grams Are Created Equal: A Study in Authorship Attribution2015
- A Straightforward Author Profiling Approach in MapReduce2014
- Optimisation of Character n-gram Profiles Method for Intrinsic Plagiarism Detection2014
- Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary ApproachPLOS ONE, 2013
- Computational methods in authorship attributionJournal of the American Society for Information Science and Technology, 2008
- WORDS VERSUS CHARACTER N-GRAMS FOR ANTI-SPAM FILTERINGInternational Journal on Artificial Intelligence Tools, 2007
- Automatically Determining an Anonymous Author’s Native Language2005
- Language independent authorship attribution using character level language models2003