Data Augmentation With Semantic Enrichment for Deep Learning Invoice Text Classification
Open Access
- 11 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 57326-57344
Abstract
Natural language processing (NLP) is a research field that provides huge potential to automate accounting tasks dealing with text data. This research studies the application of NLP in automatically categorizing invoices based on the invoice text description. The study employs semantic enrichment, data augmentation, and deep learning to address the NLP unique issues posed by the inherent short text and multi-class imbalance nature of invoice descriptions. Semantic enrichment was done using labels as an information source. Training data was artificially increased with either WordNet synonym replacement, Global Vectors for Word Representation (GloVe) word replacement, or the Bidirectional Encoder Representations from Transformers (BERT) word replacement method. Each training dataset was then supplied for training with one nondeep learning classifier and two deep learning classifiers respectively, namely Linear Support Vector Machine (LSVM), Bidirectional Long Short-Term Memory (Bi-LSTM), and BERT. Overall, the semantically enriched, WordNet augmented training set paired with the BERT classifier yielded the best results, successfully preserving semantics, reducing noise and overfitting while improving accuracy per class, achieving an increase of performance up to 20 percentage points (ppts) for macro F1 score and 6.7 ppts for accuracy.Keywords
Funding Information
- King Saud University, Riyadh, Saudi Arabia, through the Researchers Supporting Project (RSP2024R244)
This publication has 94 references indexed in Scilit:
- Data augmentation in natural language processing: a novel text generation approach for long and short text classifiersInternational Journal of Machine Learning and Cybernetics, 2022
- A deep learning approach in predicting products’ sentiment ratings: a comparative analysisThe Journal of Supercomputing, 2021
- Chatbot Interaction with Artificial Intelligence: human data augmentation with T5 and language transformer ensemble for text classificationJournal of Ambient Intelligence and Humanized Computing, 2021
- Lexical data augmentation for sentiment analysisJournal of the Association for Information Science and Technology, 2021
- Term weighting scheme for short-text classification: Twitter corpusesNeural Computing and Applications, 2018
- Evolutionary rule-based systems for imbalanced data setsSoft Computing, 2008
- Combining data and text mining techniques for analysing financial reportsIntelligent Systems in Accounting, Finance and Management, 2004
- The Nature of Statistical Learning Theory1995
- The past thirty years in information retrievalJournal of the American Society for Information Science, 1987
- Automatic query adjustment in document retrievalInformation Processing & Management, 1977