Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1
Authors:
- Julie AyreJAJulie Ayre
- Carissa BonnerCBCarissa Bonner
- Danielle M MuscatDMDanielle M Muscat
- Adam G DunnADAdam G Dunn
- Eliza HarrisonEHEliza Harrison
- Jason DalmazzoJDJason Dalmazzo
- Dana MouwadDMDana Mouwad
- Parisa AslaniParisa Aslani
- Heather L ShepherdHSHeather L Shepherd
- Kirsten J McCafferyKMKirsten J McCaffery
Open Access
- 14 February 2023
- journal article
- research article
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 7 (1) , e40645
Abstract
Producing health information that people can easily understand is challenging and time-consuming. Existing guidance is often subjective and lacks specificity. With advances in software that reads and analyzes text, there is an opportunity to develop tools that provide objective, specific, and automated guidance on the complexity of health information. This paper outlines the development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor, an automated tool to facilitate the implementation of health literacy guidelines for the production of easy-to-read written health information. Target users were any person or organization that develops consumer-facing education materials, with or without prior experience with health literacy concepts. Anticipated users included health professionals, staff, and government and nongovernment agencies. To develop this tool, existing health literacy and relevant writing guidelines were collated. Items amenable to programmable automated assessment were incorporated into the Editor. A set of natural language processing methods were also adapted for use in the SHeLL Editor, though the approach was primarily procedural (rule-based). As a result of this process, the Editor comprises 6 assessments: readability (school grade reading score calculated using the Simple Measure of Gobbledygook (SMOG)), complex language (percentage of the text that contains public health thesaurus entries, words that are uncommon in English, or acronyms), passive voice, text structure (eg, use of long paragraphs), lexical density and diversity, and person-centered language. These are presented as global scores, with additional, more specific feedback flagged in the text itself. Feedback is provided in real-time so that users can iteratively revise and improve the text. The design also includes a “text preparation” mode, which allows users to quickly make adjustments to ensure accurate calculation of readability. A hierarchy of assessments also helps users prioritize the most important feedback. Lastly, the Editor has a function that exports the analysis and revised text. The SHeLL Health Literacy Editor is a new tool that can help improve the quality and safety of written health information. It provides objective, immediate feedback on a range of factors, complementing readability with other less widely used but important objective assessments such as complex and person-centered language. It can be used as a scalable intervention to support the uptake of health literacy guidelines by health services and providers of health information. This early prototype can be further refined by expanding the thesaurus and leveraging new machine learning methods for assessing the complexity of the written text. User-testing with health professionals is needed before evaluating the Editor’s ability to improve the health literacy of written health information and evaluating its implementation into existing Australian health services.Keywords
This publication has 35 references indexed in Scilit:
- Health literacy and the Internet: a study on the readability of Australian online health informationAustralian and New Zealand Journal of Public Health, 2015
- Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient informationPatient Education and Counseling, 2014
- The CDC Clear Communication Index Is a New Evidence-Based Tool to Prepare and Review Health InformationHealth Promotion Practice, 2014
- Assessing readability formula differences with written health information materials: Application, results, and recommendationsResearch in Social and Administrative Pharmacy, 2013
- Diabetes Australia position statement. A new language for diabetes: Improving communications with and about people with diabetesDiabetes Research and Clinical Practice, 2012
- Developing and testing the health literacy universal precautions toolkitNursing Outlook, 2011
- MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessmentBehavior Research Methods, 2010
- A linguistic framework for assessing the quality of written patient information: its use in assessing methotrexate information for rheumatoid arthritisHealth Education Research, 2004
- Teaching Patients with Low Literacy SkillsAJN, American Journal of Nursing, 1996
- Readability formulas: Cautions and criteriaPatient Education and Counseling, 1991