Exploring trust in AI-supported military teams using sentiment analysis
Authors:
- Murat KucukosmanogluMKMurat Kucukosmanoglu
- Craig JohnsonCJCraig Johnson
- Kimberly PollardKPKimberly Pollard
- David ChhanDCDavid Chhan
- Shan LakhmaniSLShan Lakhmani
- Daniel ForsterDFDaniel Forster
- Sarah ConklinSCSarah Conklin
- Justin BrooksJBJustin Brooks
- H. Philip CrowellHCH. Philip Crowell
- Andrea KrausmanAKAndrea Krausman
- 31 December 2025
- journal article
- Published by John Benjamins Publishing Company
- Vol. 26 (2) , 229-266
Abstract
Examining sentiment in team communications can provide information about trust among teammates. Natural language processing (NLP) models provide an efficient means of sentiment analysis. However, military teams and other professional teams use language that differs from what NLP models are trained on, leading to potentially inaccurate sentiment analysis. This study investigates the novel application of two advanced NLP models, DistilBERT and GPT-2, for sentiment analysis of expert military teams conducting AI-supported combat missions in a high fidelity simulation environment. Our fine-tuning process resulted in improved sentiment classification accuracy. The sentiment measures also correlated with measures of team trust and trust in the AI systems, providing valuable insight into the relationship between sentiment and trust in human-AI teaming scenarios. The generalized approach we describe may be useful for adapting sentiment analysis and NLP techniques to military teams, and may help measure trust dynamics and team states in human machine integrated teams.Keywords
This publication has 55 references indexed in Scilit:
- Affective Responses to Trust Violations in a Human-Autonomy Teaming Context: Humans Versus RobotsInternational Journal of Social Robotics, 2023
- Approaches for assessing communication in human-autonomy teamsHuman-Intelligent Systems Integration, 2021
- Artificial intelligence for decision support systems in the field of operations research: review and future scope of researchAnnals of Operations Research, 2021
- Conversational robo advisors as surrogates of trust: onboarding experience, firm perception, and consumer financial decision makingJournal of the Academy of Marketing Science, 2020
- Measuring team trust: A critical and meta‐analytical reviewJournal of Organizational Behavior, 2020
- Towards a Theory of Longitudinal Trust Calibration in Human–Robot TeamsInternational Journal of Social Robotics, 2019
- Exploration of social media for sentiment analysis using deep learningSoft Computing, 2019
- Sentiment Analysis on Online Product Reviews2019
- Trust in work teams: An integrative review, multilevel model, and future directionsJournal of Organizational Behavior, 2017
- Coefficient Alpha and the Internal Structure of TestsPsychometrika, 1951