A Meaningful Information Extraction System for Interactive Analysis of Documents
Authors:
- 1 September 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 92-99
Abstract
This paper is related to a project aiming at discovering weak signals from different streams of information, possibly sent by whistleblowers. The study presented in this paper tackles the particular problem of clustering topics at multi-levels from multiple documents, and then extracting meaningful descriptors, such as weighted lists of words for document representations in a multi-dimensions space. In this context, we present a novel idea which combines Latent Dirichlet Allocation and Word2vec (providing a consistency metric regarding the partitioned topics) as potential method for limiting the "a priori" number of cluster K usually needed in classical partitioning approaches. We proposed 2 implementations of this idea, respectively able to: (1) finding the best K for LDA in terms of topic consistency; (2) gathering the optimal clusters from different levels of clustering. We also proposed a non-traditional visualization approach based on a multi-agents system which combines both dimension reduction and interactivity.Keywords
This publication has 16 references indexed in Scilit:
- Novelty-focused weak signal detection in futuristic data: Assessing the rarity and paradigm unrelatedness of signalsTechnological Forecasting and Social Change, 2017
- A Survey of Topic Modeling in Text MiningInternational Journal of Advanced Computer Science and Applications, 2015
- Linguistic Regularities in Sparse and Explicit Word Representations2014
- Weak signal identification with semantic web miningExpert Systems with Applications, 2013
- Detecting weak signals for long-term business opportunities using text mining of Web newsExpert Systems with Applications, 2012
- Joint latent topic models for text and citations2008
- The future sign and its three dimensionsFutures, 2008
- Dynamic topic models2006
- An internet‐based approach to environmental scanning in marketing planningMarketing Intelligence & Planning, 2005
- Managing Strategic Surprise by Response to Weak SignalsCalifornia Management Review, 1975