Impact of Iterative Development and Beta-Testing on the Usability and Acceptability of a Novel Just-in-Time Adaptive Digital Physical Activity Intervention
Open Access
- 1 October 2025
- journal article
- Published by Human Kinetics in Journal of Physical Activity and Health
- Vol. 22 (10) , 1315-1321
Abstract
Background: The search for cost-effective population-based physical activity interventions continues. Therefore, we developed a novel just-in-time adaptive digital assistant supported by machine learning (ie, MoveMentor). Beta-testing is essential to evaluate both technical performance and user acceptance. The aim of this study was to assess app usability, acceptability, and technical performance through iterative rounds of beta-testing. Methods: Insufficiently active people (age: 39.8 [10.2]; 86% female) participated in 2 rounds of beta-testing (round 1, n = 112; round 2, n = 41). Participants downloaded the digital assistant app onto their phone to use during the study period (round 1: 12 wk, round 2: 4 wk). Participants were asked complete at least 4 educational and 5 chat conversations, rate over 50 notifications, and complete an online follow-up survey at week 4 examining aspects of app usability and acceptability. Descriptive statistics and t tests were used to analyze outcomes. Results: Across both rounds, the app demonstrated good overall usability scores (System Usability Scale: 75.3 out of 100) but lower usefulness ratings. Round 2 participants showed increased engagement with features including action plans (P < .001), educational conversations (P < .001), and personalization features (P < .001), and they appreciated the educational conversations more (P < .05). Technical issues including data syncing problems and chat limitations persisted across both rounds. The notification system received mixed feedback, though customization options in round 2 reduced complaints (12.2%–7.3%). Conclusions: The app demonstrated good acceptability and usability but low usefulness. The iterative beta-testing successfully identified areas for improvement and enabled meaningful enhancements to content and user engagement features. While some technical challenges persisted, the beta-testing provided clear direction for ongoing improvements.Keywords
This publication has 22 references indexed in Scilit:
- Increasing physical activity using an just-in-time adaptive digital assistant supported by machine learning: A novel approach for hyper-personalised mHealth interventionsJournal of Biomedical Informatics, 2023
- Acceptability, usefulness, and satisfaction with a web-based video-tailored physical activity intervention: The TaylorActive randomized controlled trialJournal of Sport and Health Science, 2021
- Acceptability of digital health interventions: embracing the complexityTranslational Behavioral Medicine, 2021
- Assessing usability of eHealth technology: A comparison of usability benchmarking instrumentsInternational Journal of Medical Informatics, 2019
- Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesisTranslational Behavioral Medicine, 2016
- The economic burden of physical inactivity: a global analysis of major non-communicable diseasesThe Lancet, 2016
- Face-to-face interventions for promoting physical activity2013
- STARE-HI—Statement on reporting of evaluation studies in Health InformaticsInternational Journal of Medical Informatics, 2009
- Test-retest reliability of four physical activity measures used in population surveysJournal of Science and Medicine in Sport, 2004
- Acceptability and feasibility of a computer-tailored physical activity intervention using stages of change: project FAITHHealth Education Research, 2003