Prototype Theory Meets Word Embedding: A Novel Approach for Text Categorization via Granular Computing
Open Access
- 23 March 2023
- journal article
- research article
- Published by Springer Nature in Cognitive Computation
- Vol. 15 (3) , 976-997
Abstract
The problem of the information representation and interpretation coming from senses by the brain has plagued scientists for decades. The same problems, from a different perspective, hold in automated Pattern Recognition systems. Specifically, in solving various NLP tasks, an ever better and richer semantic representation of text as a set of features is needed and a plethora of text embedding techniques in algebraic spaces are continuously provided by researchers. These spaces are well suited to be conceived as conceptual spaces in light of the Gärdenfors’s Conceptual Space theory, which, within the Cognitive Science paradigm, seeks a geometrization of thought that bridges the gap between an associative lower level and a symbolic higher level in which information is organized and processed and where inductive reasoning is appropriate. Granular Computing can offer the toolbox for granulating text that can be represented by more abstract entities than words, offering a good hierarchical representation of the text embedded in an algebraic space driving Machine Learning applications, specifically, in text mining tasks. In this paper, the Conceptual Space Theory, the Granular Computing approach and Machine Learning are bound in a novel common framework for solving some text categorization tasks with both standard classifiers suited for working with $$\mathbb {R}^n$$ vectors and a Recurrent Neural Network (RNN) — an LSTM — able to deal with sequences. Instead of working with word vectors, the algorithms process more abstract entities (concepts), where patterns, in a first approach, are obtained through the construction of a symbolic histogram starting from a suitable set of information granules, representing a document as a distribution of concepts. For the RNN case, as a further novelty, a text is represented as a random walk over prototypes within the conceptual space synthesized over a suitable text embedding procedure. A comparison of the performance and a critical discussion are offered for both a neural embedding technique and the well-known LSA, showing how the conceptual level leads also to Knowledge Discovery applications.
Keywords
Funding Information
- Università degli Studi di Roma La Sapienza
This publication has 67 references indexed in Scilit:
- Extracting prototypes from exemplars What can corpus data tell us about concept representation?Cognitive Linguistics, 2013
- On the origin of long-range correlations in textsProceedings of the National Academy of Sciences, 2012
- Mining data with random forests: A survey and results of new testsPattern Recognition, 2011
- Complex Systems and Applied LinguisticsInternational Journal of Applied Linguistics, 2007
- Towards context sensitive information inferenceJournal of the American Society for Information Science and Technology, 2003
- On the algorithmic implementation of stochastic discriminationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- An introduction to latent semantic analysisDiscourse Processes, 1998
- Least squares quantization in PCMIEEE Transactions on Information Theory, 1982
- A vector space model for automatic indexingCommunications of the ACM, 1975
- Wittgenstein's Philosophical InvestigationsPhilosophical Review, 1954