Probabilistic Prediction of Nonadherence to Psychiatric Disorder Medication from Mental Health Forum Data: Developing and Validating Bayesian Machine Learning Classifiers
Open Access
- 15 April 2022
- journal article
- retracted article
- Published by Wiley in Computational Intelligence and Neuroscience
- Vol. 2022, 1-15
Abstract
Background. Medication nonadherence represents a major burden on national health systems. According to the World Health Organization, increasing medication adherence may have a greater impact on public health than any improvement in specific medical treatments. More research is needed to better predict populations at risk of medication nonadherence. Objective. To develop clinically informative, easy-to-interpret machine learning classifiers to predict people with psychiatric disorders at risk of medication nonadherence based on the syntactic and structural features of written posts on health forums. Methods. All data were collected from posts between 2016 and 2021 on mental health forum, administered by Together 4 Change, a long-running not-for-profit organisation based in Oxford, UK. The original social media data were annotated using the Tool for the Automatic Analysis of Syntactic Sophistication and Complexity (TAASSC) system. Through applying multiple feature optimisation techniques, we developed a best-performing model using relevance vector machine (RVM) for the probabilistic prediction of medication nonadherence among online mental health forum discussants. Results. The best-performing RVM model reached a mean AUC of 0.762, accuracy of 0.763, sensitivity of 0.779, and specificity of 0.742 on the testing dataset. It outperformed competing classifiers with more complex feature sets with statistically significant improvement in sensitivity and specificity, after adjusting the alpha levels with Benjamini–Hochberg correction procedure. Discussion. We used the forest plot of multiple logistic regression to explore the association between written post features in the best-performing RVM model and the binary outcome of medication adherence among online post contributors with psychiatric disorders. We found that increased quantities of 3 syntactic complexity features were negatively associated with psychiatric medication adherence: “dobj_stdev” (standard deviation of dependents per direct object of nonpronouns) (OR, 1.486, 95% CI, 1.202–1.838, ), “cl_av_deps” (dependents per clause) (OR, 1.597, 95% CI, 1.202–2.122, , 0.001), and “VP_T” (verb phrases per T-unit) (OR, 2.23, 95% CI, 1.211–4.104, , 0.010). Finally, we illustrated the clinical use of the classifier with Bayes’ monograph which gives the posterior odds and their 95% CI of positive (nonadherence) versus negative (adherence) cases as predicted by the best-performing classifier. The odds ratio of the posterior probability of positive cases was 3.9, which means that around 10 in every 13 psychiatric patients with a positive result as predicted by our model were following their medication regime. The odds ratio of the posterior probability of true negative cases was 0.4, meaning that around 10 in every 14 psychiatric patients with a negative test result after screening by our classifier were not adhering to their medications. Conclusion. Psychiatric medication nonadherence is a large and increasing burden on national health systems. Using Bayesian machine learning techniques and publicly accessible online health forum data, our study illustrates the viability of developing cost-effective, informative decision aids to support the monitoring and prediction of patients at risk of medication nonadherence.Keywords
Funding Information
- National Natural Science Foundation of China (61772146, 2021A1515011339)
- Natural Science Foundation of Guangdong Province (61772146, 2021A1515011339)
This publication has 48 references indexed in Scilit:
- Predicting Glaucomatous Progression in Glaucoma Suspect Eyes Using Relevance Vector Machine Classifiers for Combined Structural and Functional MeasurementsInvestigative Ophthalmology & Visual Science, 2012
- Understanding diagnostic tests 2: likelihood ratios, pre‐ and post‐test probabilities and their use in clinical practiceActa Paediatrica, 2007
- Literacy, self-efficacy, and HIV medication adherencePatient Education and Counseling, 2007
- Adherence to MedicationNew England Journal of Medicine, 2005
- Variations in Patients’ Adherence to Medical RecommendationsMedical Care, 2004
- Social Support and Patient Adherence to Medical Treatment: A Meta-Analysis.Health Psychology, 2004
- Using Bayes' nomogram to help interpret odds ratiosBMJ Evidence-Based Medicine, 2003
- The beliefs about medicines questionnaire: The development and evaluation of a new method for assessing the cognitive representation of medicationPsychology & Health, 1999
- An Assessment of Clinically Useful Measures of the Consequences of TreatmentNew England Journal of Medicine, 1988
- NOMOGRAM FOR PREDICTIVE VALUES AND EFFICIENCIES OF TESTSThe Lancet, 1985