Enhancing User Trust and Interpretability in AI-Driven Feature Request Detection for Mobile App Reviews: An Explainable Approach
Open Access
- 14 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 114023-114045
Abstract
Mobile app developers struggle to prioritize updates by identifying feature requests within user reviews. While machine learning models can assist, their complexity often hinders transparency and trust. This paper presents an explainable Artificial Intelligence (AI) approach that combines advanced explanation techniques with engaging visualizations to address this issue. Our system integrates a bidirectional Long Short-Term Memory (BiLSTM) model with attention mechanisms, enhanced by Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). We evaluate this approach on a diverse dataset of 150,000 app reviews, achieving an F1 score of 0.82 and 89% accuracy, significantly outperforming baseline Support Vector Machine (F1: 0.66) and Convolutional Neural Network (CNN) (F1: 0.72) models. Our empirical user studies with developers demonstrate that our explainable approach improves trust (27%) when explanations are provided and correct interpretation (73%). The system’s interactive visualizations allowed developers to validate predictions, with over 80% overlap between model-highlighted phrases and human annotations for feature requests. These findings highlight the importance of integrating explainable AI into real-world software engineering workflows. The paper’s results and future directions provide a promising approach for feature request detection in app reviews to create more transparent, trustworthy, and effective AI systems.Keywords
Funding Information
- National Science and Technology Council (NSC 111-2410-H-167-005-MY2, NSC112-2634-F-005-001-MBK)
This publication has 93 references indexed in Scilit:
- An Overview of Explainable AI Methods, Forms and Frameworks2022
- Approximation of SHAP Values for Randomized Tree Ensembles2022
- Stakeholder-Centric Clustering Methods for Conflict Resolution in the Requirements Engineering Process2022
- Opinion mining for app reviews: an analysis of textual representation and predictive modelsAutomated Software Engineering, 2021
- CRAN: An Hybrid CNN-RNN Attention-Based Model for Arabic Machine Translation2021
- Principles of Explainable Artificial Intelligence2021
- Explainability as a non-functional requirement: challenges and recommendationsRequirements Engineering, 2020
- Mining User Opinions to Support Requirement Engineering: An Empirical Study2020
- Profiling users via their reviews: an extended systematic mapping studySoftware and Systems Modeling, 2020
- On the automatic classification of app reviewsRequirements Engineering, 2016