What You Read Is What You Classify: Highlighting Attributions to Text and Text-like Inputs
Open Access
Abstract
At present, there are no easily understood explainable artificial intelligence (AI) methods for discrete token inputs, like text. Most explainable AI techniques do not extend well to token sequences, where both local and global features matter, because state-of-the-art models, like transformers, tend to focus on global connections. Therefore, existing explainable AI algorithms fail by (i) identifying disparate tokens of importance, or (ii) assigning a large number of tokens a low value of importance. This method for explainable AI for tokens-based classifiers generalizes a mask-based explainable AI algorithm designed originally for images. It starts with an Explainer neural network that is trained to create masks to hide information not relevant for classification. Then, the Hadamard product of the mask and the continuous values of the classifier’s embedding layer is taken and passed through the classifier, changing the magnitude of the embedding vector but keeping the orientation unchanged. The Explainer is trained for a taxonomic classifier for nucleotide sequences and it is shown that the masked segments are less relevant to classification than the unmasked ones. This method focused on the importance the token as a whole (i.e., a segment of the input sequence), producing a human-readable explanation.Keywords
All Related Versions
Funding Information
- Centers for Disease Control and Prevention (75D30124C20202)
This publication has 33 references indexed in Scilit:
- Classifying human vs. AI text with machine learning and explainable transformer modelsScientific Reports, 2025
- Evaluating the Effectiveness of Parameter-Efficient Fine-Tuning in Genomic Classification TasksbioRxiv, 2025
- Nucleotide Transformer: building and evaluating robust foundation models for human genomicsNature Methods, 2024
- Fine-Grained Multilingual Hate Speech Detection Using Explainable AI and TransformersIEEE Access, 2024
- Advanced sentence-embedding method considering token importance based on explainable artificial intelligence and text summarization modelNeurocomputing, 2023
- Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision taskComputers in Human Behavior, 2022
- On Exploring Attention-based Explanation for Transformer Models in Text Classification2021
- BERTax: taxonomic classification of DNA sequences with Deep Neural NetworksbioRxiv, 2021
- The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AIInternational Journal of Human-Computer Studies, 2021
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019