Stopwords in Technical Language Processing
Preprint
Abstract
There are increasingly applications of natural language processing techniques for information retrieval, indexing and topic modelling in the engineering contexts. A standard component of such tasks is the removal of stopwords, which are uninformative components of the data. While researchers use readily available stopword lists which are derived for general English language, the technical jargon of engineering fields contains their own highly frequent and uninformative words and there exists no standard stopword list for technical language processing applications. Here we address this gap by rigorously identifying generic, insignificant, uninformative stopwords in engineering texts beyond the stopwords in general texts, based on the synthesis of alternative data-driven approaches, and curating a stopword list ready for technical language processing applications.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- TechNet: Technology semantic network based on patent dataExpert Systems with Applications, 2020
- Data-Driven Concept Network for Inspiring Designers’ Idea GenerationJournal of Computing and Information Science in Engineering, 2020
- A universal information theoretic approach to the identification of stopwordsNature Machine Intelligence, 2019
- Product concept evaluation and selection using data mining and domain ontology in a crowdsourcing environmentAdvanced Engineering Informatics, 2015
- “Term clumping” for technical intelligence: A case study on dye-sensitized solar cellsTechnological Forecasting and Social Change, 2014
- TOWARDS THE QUANTIFICATION OF THE SEMANTIC INFORMATION ENCODED IN WRITTEN LANGUAGEAdvances in Complex Systems, 2010
- Understanding and customizing stopword lists for enhanced patent mappingWorld Patent Information, 2007
- A stop list for general textACM SIGIR Forum, 1989
- Coefficient Alpha and the Internal Structure of TestsPsychometrika, 1951