An interpretable method for automated classification of spoken transcripts and written text
Open Access
- 4 May 2023
- journal article
- research article
- Published by Springer Nature in Evolutionary Intelligence
- Vol. 17 (1) , 609-621
Abstract
We investigate the differences between spoken language (in the form of radio show transcripts) and written language (Wikipedia articles) in the context of text classification. We present a novel, interpretable method for text classification, involving a linear classifier using a large set of $$n-$$ gram features, and apply it to a newly generated data set with sentences originating either from spoken transcripts or written text. Our classifier reaches an accuracy less than 0.02 below that of a commonly used classifier (DistilBERT) based on deep neural networks (DNNs). Moreover, our classifier has an integrated measure of confidence, for assessing the reliability of a given classification. An online tool is provided for demonstrating our classifier, particularly its interpretable nature, which is a crucial feature in classification tasks involving high-stakes decision-making. We also study the capability of DistilBERT to carry out fill-in-the-blank tasks in either spoken or written text, and find it to perform similarly in both cases. Our main conclusion is that, with careful improvements, the performance gap between classical methods and DNN-based methods may be reduced significantly, such that the choice of classification method comes down to the need (if any) for interpretability.
Keywords
Funding Information
- Chalmers University of Technology
This publication has 16 references indexed in Scilit:
- Handbook on Computer Learning and Intelligence2021
- Deep Learning--based Text ClassificationACM Computing Surveys, 2021
- Fooling LIME and SHAP2020
- A Diagnostic Study of Explainability Techniques for Text Classification2020
- A smart System for Fake News Detection Using Machine Learning2019
- Stop explaining black box machine learning models for high stakes decisions and use interpretable models insteadNature Machine Intelligence, 2019
- Chinese Computational Linguistics2019
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)IEEE Access, 2018
- "What is relevant in a text document?": An interpretable machine learning approachPLOS ONE, 2017
- "Why Should I Trust You?"2016