Felix Greaves

Background: Clinically coded long COVID cases in electronic health records (EHRs) are incomplete, despite reports of rising cases of long COVID. Aim: To determine patient characteristics associated with clinically coded long COVID. Design & setting: With the approval of NHS England, we conducted a cohort study using EHRs within the OpenSAFELY-TPP platform in England, to study patient characteristics associated with clinically coded long COVID from 29 January 2020 to 31 March 2022. Method: We summarised the distribution of characteristics for people with clinically coded long COVID. We estimated age–sex adjusted hazard ratios (aHRs) and fully aHRs for coded long COVID. Patient characteristics included demographic factors, and health behavioural and clinical factors. Results: Among 17 986 419 adults, 36 886 (0.21%) were clinically coded with long COVID. Patient characteristics associated with coded long COVID included female sex, younger age (aged <60 years), obesity, living in less deprived areas, ever smoking, greater consultation frequency, and history of diagnosed asthma, mental health conditions, pre-pandemic post-viral fatigue, or psoriasis. These associations were attenuated following two doses of COVID-19 vaccines compared with before vaccination. Differences in the predictors of coded long COVID between the pre-vaccination and post-vaccination cohorts may reflect the different patient characteristics in these two cohorts rather than the vaccination status. Incidence of coded long COVID was higher in those with hospitalised COVID-19 than with those with non-hospitalised COVID-19. Conclusion: We identified variation in coded long COVID by patient characteristic. Results should be interpreted with caution as long COVID was likely under-recorded in EHRs.

Background: Digital interventions can be effective for reducing alcohol consumption. However, most digital interventions that have been evaluated are websites and there is little evidence on the effectiveness of smartphone apps, especially in a United Kingdom context. We developed an evidence- and theory-informed app, Drink Less, to help increasing-and-higher-risk drinkers (Alcohol Use Disorders Identification Test score ≥ 8) reduce their alcohol consumption. Objective: To evaluate the effectiveness of Drink Less for reducing alcohol consumption compared with usual digital care in the United Kingdom. Design: Two-arm, double-blind, parallel-group, randomised controlled trial with 1 : 1 group allocation and an embedded process evaluation, with 6-month follow-up. Setting: Remotely conducted among participants living in the United Kingdom, recruited from July 2020 to March 2022. Participants: Five thousand six hundred and two increasing-and-higher-risk drinkers aged 18+ who had access to an iPhone operating system device and wanted to drink less alcohol. Interventions: Participants were recommended to use the intervention (Drink Less) or recommended the comparator (National Health Service alcohol advice web page). Drink Less is an app-based intervention to help increasing-and-higher-risk drinkers reduce their alcohol consumption. It consists of evidence-based modules (e.g. goal setting, self-monitoring) and was systematically and transparently developed and refined. The National Health Service alcohol advice web page was considered usual digital care and provides tips on cutting down. Main outcome measures: The primary outcome was self-reported weekly alcohol consumption at 6-month follow-up (derived from the extended Alcohol Use Disorders Identification Test – Consumption), adjusted for baseline alcohol consumption. Results: The retention rate at 6-month follow-up was 80%. The data were not missing completely at random with differences detected in educational qualifications, occupation and income, indicating that multiple imputation was the most appropriate analytic approach. This found that Drink Less resulted in a 2.00 United Kingdom unit greater weekly reduction (95% confidence interval −3.76 to −0.24) at 6-month follow-up compared with the National Health Service alcohol advice web page. Compared with the National Health Service alcohol advice web page, Drink Less cost an additional £1.28 per user, when including the sunk costs (already incurred and cannot be recovered), but saved £0.04 per user when considering only the annual maintenance costs. Drink Less costs only an extra £0.64 per additional weekly unit of alcohol reduction, and may be cost saving if sufficient people use the app to cover the sunk costs. There was no statistically significant difference in quality-adjusted life-years between the two groups. Limitations: This trial relied on retrospective self-reported alcohol consumption. Results from the pre-registered sensitivity analysis of multiple imputation were inconsistent with those from the pre-registered primary analysis (a conservative approach to missing data where non-responders were assumed to be drinking at baseline levels), which found a non-significant weekly reduction of 0.98 units (95% confidence interval −2.67 to 0.70) in the intervention compared with comparator group. Multiple imputation was recommended by the independent Data Monitoring Committee based on the pattern of missing data. Conclusions: Drink Less appears effective for reducing alcohol consumption among increasing-and-higher-risk drinkers compared with the National Health Service alcohol advice web page in the United Kingdom, and may be cost saving if widely used in the population. Future work: Drink Less is in a strong position to be promoted widely and provide inexpensive support to increasing-and-higher-risk drinkers in the United Kingdom. Future work should investigate different promotion strategies and ways of implementing the app within healthcare settings and adapting it for other countries. Funding: This synopsis presents independent research funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme as award number NIHR127651.

Background: The adoption of patient portals, such as the National Health Service (NHS) App in England, may improve patient engagement in health care. However, concerns remain regarding differences across sociodemographic groups in the uptake and use of various patient portal features, which have not been fully explored. Understanding the use of various functions across diverse populations is essential to ensure any benefits are equally distributed across the population. Objective: This study aims to explore differences in the use of NHS App features across age, sex, deprivation, ethnicity, long-term health care needs, and general practice (GP) size categories. Methods: We used weekly NHS App use data from the NHS App dashboard for 6386 GPs in England from March 2020 to June 2022. Negative binomial regression models explored variations in weekly rates of NHS App features used (registrations, log-ins, prescriptions ordered, medical record views, and appointments booked). Outcomes were measured as weekly rates per 1000 GP-registered patients, and we conducted separate models for each outcome. Regression models included all covariates mentioned above and produced incident rate ratios, which we present here as relative percentages for ease of interpretation. GP-level covariate data on sociodemographic variables were used as categorical variables in 5 groups for deprivation (Q1=least deprived practices and Q5=most deprived practices) and 4 groups for all other variables (Q1=least deprived practices and Q4=most deprived practices). Results: We found variations in the use of different features overall and across sociodemographic categories. Fully adjusted regression models found lower use of features overall in more deprived practices (eg, Q5 vs Q1: registrations=–34%, log-ins=–34.9%, appointments booked=–39.7%, medical record views=–32.3%, and prescriptions ordered=–9.9%; P<.001). Practices with greater proportions of male patients also had lower levels of NHS App use (eg, Q4 vs Q1: registration=–7.1%, log-in=–10.4%, and appointments booked=–36.4%; P<.001). Larger practices had an overall higher use of some NHS App features (eg, Q4 vs Q1: registration=3.2%, log-ins=11.7%, appointments booked=73.4%, medical record views=23.9%, and prescriptions ordered=20.7%; P<.001), as well as those with greater proportions of White patients (eg, Q4 vs Q1: registration=1.9%, log-ins=9.1%, appointments booked=14.1%, medical record views=28.7%, and prescriptions ordered=130.4%; P<.001). Use patterns varied for practices with greater proportions of patients with long-term health care needs (eg, Q4 vs Q1: registrations=–3.6%, appointments booked=–20%, and medical record views=6%; P≤.001). Conclusions: This study highlights that the use of the NHS App features varied across sociodemographic groups. In particular, it is used less by people living in more deprived areas. Tailored interventions and patient support are required to ensure that any benefits from the NHS App are spread equally throughout the population.

Background: The extent to which interventions are perceived as acceptable to users impacts engagement and efficacy. Objective: In this study, we evaluated the acceptability of (1) the smartphone app Drink Less (intervention) and (2) the National Health Service (NHS) alcohol advice web page (usual digital care and comparator) among adult drinkers in the United Kingdom participating in a randomized controlled trial evaluating the effectiveness of the Drink Less app. Methods: A subsample of 26 increasing- and higher-risk drinkers (Alcohol Use Disorders Identification Test score≥8) assigned to the intervention group (Drink Less; n=14, 54%; female: n=10, 71%; age: 22-72 years; White: n=9, 64%) or usual digital care group (NHS alcohol advice web page; n=12, 46%; female: n=5, 42%; age: 23-68 years: White: n=9, 75%) took part in semistructured interviews. The interview questions were mapped on to the 7 facets of acceptability according to the Theoretical Framework of Acceptability: affective attitude, burden, perceived effectiveness, ethicality, intervention coherence, opportunity costs, and self-efficacy. Alongside these constructs, we also included a question on perceived personal relevance, which previous research has linked to acceptability and engagement. Framework and thematic analysis of data was undertaken. Results: The Drink Less app was perceived as being ethical, easy, user-friendly, and effective for the period the app was used. Participants reported particularly liking the tracking and feedback sections of the app, which they reported increased personal relevance and which resulted in positive affect when achieving their goals. They reported no opportunity cost. Factors such as negative affect when not meeting goals and boredom led to disengagement in the longer term for some participants. The NHS alcohol advice web page was rated as being easy and user-friendly with no opportunity costs. However, the information presented was not perceived as being personally relevant or effective in changing drinking behavior. Most participants reported neutral or negative affect, most participants thought the alcohol advice web page was accessible, and some participants reported ethical concerns around the availability of suggested resources. Some participants reported that it had acted as a starting point or a signpost to other resources. Participants in both groups discussed motivation to change and contextual factors such as COVID-19 lockdowns, which influenced their perceived self-efficacy regardless of their assigned intervention. Conclusions: Drink Less appears to be an acceptable digital intervention among the recruited sample. The NHS alcohol advice web page was generally considered unacceptable as a stand-alone intervention among the recruited sample, although it may signpost and help people access other resources and interventions.

During a quit attempt, cues from a smoker's environment are a major cause of brief smoking lapses, which increase the risk of relapse. Quit Sense is a theory-guided Just-In-Time Adaptive Intervention smartphone app, providing smokers with the means to learn about their environmental smoking cues and provides 'in the moment' support to help them manage these during a quit attempt. To undertake a feasibility randomised controlled trial to estimate key parameters to inform a definitive randomised controlled trial of Quit Sense. A parallel, two-arm randomised controlled trial with a qualitative process evaluation and a 'Study Within A Trial' evaluating incentives on attrition. The research team were blind to allocation except for the study statistician, database developers and lead researcher. Participants were not blind to allocation. Online with recruitment, enrolment, randomisation and data collection (excluding manual telephone follow-up) automated through the study website. Smokers (323 screened, 297 eligible, 209 enrolled) recruited via online adverts on Google search, Facebook and Instagram. Participants were allocated to 'usual care' arm (n = 105; text message referral to the National Health Service SmokeFree website) or 'usual care' plus Quit Sense (n = 104), via a text message invitation to install the Quit Sense app. Follow-up at 6 weeks and 6 months post enrolment was undertaken by automated text messages with an online questionnaire link and, for non-responders, by telephone. Definitive trial progression criteria were met if a priori thresholds were included in or lower than the 95% confidence interval of the estimate. Measures included health economic and outcome data completion rates (progression criterion #1 threshold: ≥ 70%), including biochemical validation rates (progression criterion #2 threshold: ≥ 70%), recruitment costs, app installation (progression criterion #3 threshold: ≥ 70%) and engagement rates (progression criterion #4 threshold: ≥ 60%), biochemically verified 6-month abstinence and hypothesised mechanisms of action and participant views of the app (qualitative). Self-reported smoking outcome completion rates were 77% (95% confidence interval 71% to 82%) and health economic data (resource use and quality of life) 70% (95% CI 64% to 77%) at 6 months. Return rate of viable saliva samples for abstinence verification was 39% (95% CI 24% to 54%). The per-participant recruitment cost was £19.20, which included advert (£5.82) and running costs (£13.38). In the Quit Sense arm, 75% (95% CI 67% to 83%; 78/104) installed the app and, of these, 100% set a quit date within the app and 51% engaged with it for more than 1 week. The rate of 6-month biochemically verified sustained abstinence, which we anticipated would be used as a primary outcome in a future study, was 11.5% (12/104) in the Quit Sense arm and 2.9% (3/105) in the usual care arm (estimated effect size: adjusted odds ratio = 4.57, 95% CIs 1.23 to 16.94). There was no evidence of between-arm differences in hypothesised mechanisms of action. Three out of four progression criteria were met. The Study Within A Trial analysis found a £20 versus £10 incentive did not significantly increase follow-up rates though reduced the need for manual follow-up and increased response speed. The process evaluation identified several potential pathways to abstinence for Quit Sense, factors which led to disengagement with the app, and app improvement suggestions. Biochemical validation rates were lower than anticipated and imbalanced between arms. COVID-19-related restrictions likely limited opportunities for Quit Sense to provide location tailored support. The trial design and procedures demonstrated feasibility and evidence was generated supporting the efficacy potential of Quit Sense. Progression to a definitive trial is warranted providing improved biochemical validation rates. This trial is registered as ISRCTN12326962. This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 17/92/31) and is published in full in Public Health Research; Vol. 12, No. 4. See the NIHR Funding and Awards website for further award information.

Background: Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation. Objective: This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England. Methods: This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence’s Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from “definitely exclude” to “definitely include,” and suggest edits. The document will be iterated between rounds based on participants’ feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote. Results: Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024. Conclusions: The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas. International Registered Report Identifier (IRRID): DERR1-10.2196/50568

BACKGROUND The adoption of patient portals, such as the National Health Service (NHS) App in England, may improve patient engagement in health care. However, concerns remain regarding differences across sociodemographic groups in the uptake and use of various patient portal features, which have not been fully explored. Understanding the use of various functions across diverse populations is essential to ensure any benefits are equally distributed across the population. OBJECTIVE This study aims to explore differences in the use of NHS App features across age, sex, deprivation, ethnicity, long-term health care needs, and general practice (GP) size categories. METHODS We used weekly NHS App use data from the NHS App dashboard for 6386 GPs in England from March 2020 to June 2022. Negative binomial regression models explored variations in weekly rates of NHS App features used (registrations, log-ins, prescriptions ordered, medical record views, and appointments booked). Outcomes were measured as weekly rates per 1000 GP-registered patients, and we conducted separate models for each outcome. Regression models included all covariates mentioned above and produced incident rate ratios, which we present here as relative percentages for ease of interpretation. GP-level covariate data on sociodemographic variables were used as categorical variables in 5 groups for deprivation (Q1=least deprived practices and Q5=most deprived practices) and 4 groups for all other variables (Q1=least deprived practices and Q4=most deprived practices). RESULTS We found variations in the use of different features overall and across sociodemographic categories. Fully adjusted regression models found lower use of features overall in more deprived practices (eg, Q5 vs Q1: registrations=–34%, log-ins=–34.9%, appointments booked=–39.7%, medical record views=–32.3%, and prescriptions ordered=–9.9%; P<.001). Practices with greater proportions of male patients also had lower levels of NHS App use (eg, Q4 vs Q1: registration=–7.1%, log-in=–10.4%, and appointments booked=–36.4%; P<.001). Larger practices had an overall higher use of some NHS App features (eg, Q4 vs Q1: registration=3.2%, log-ins=11.7%, appointments booked=73.4%, medical record views=23.9%, and prescriptions ordered=20.7%; P<.001), as well as those with greater proportions of White patients (eg, Q4 vs Q1: registration=1.9%, log-ins=9.1%, appointments booked=14.1%, medical record views=28.7%, and prescriptions ordered=130.4%; P<.001). Use patterns varied for practices with greater proportions of patients with long-term health care needs (eg, Q4 vs Q1: registrations=–3.6%, appointments booked=–20%, and medical record views=6%; P≤.001). CONCLUSIONS This study highlights that the use of the NHS App features varied across sociodemographic groups. In particular, it is used less by people living in more deprived areas. Tailored interventions and patient support are required to ensure that any benefits from the NHS App are spread equally throughout the population.

BACKGROUND Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation. OBJECTIVE This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England. METHODS This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence’s Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from “definitely exclude” to “definitely include,” and suggest edits. The document will be iterated between rounds based on participants’ feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote. RESULTS Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024. CONCLUSIONS The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas. INTERNATIONAL REGISTERED REPORT DERR1-10.2196/50568

Background: Evaluating digital interventions using remote methods enables the recruitment of large numbers of participants relatively conveniently and cheaply compared with in-person methods. However, conducting research remotely based on participant self-report with little verification is open to automated “bots” and participant deception. Objective: This paper uses a case study of a remotely conducted trial of an alcohol reduction app to highlight and discuss (1) the issues with participant deception affecting remote research trials with financial compensation; and (2) the importance of rigorous data management to detect and address these issues. Methods: We recruited participants on the internet from July 2020 to March 2022 for a randomized controlled trial (n=5602) evaluating the effectiveness of an alcohol reduction app, Drink Less. Follow-up occurred at 3 time points, with financial compensation offered (up to £36 [US $39.23]). Address authentication and telephone verification were used to detect 2 kinds of deception: “bots,” that is, automated responses generated in clusters; and manual participant deception, that is, participants providing false information. Results: Of the 1142 participants who enrolled in the first 2 months of recruitment, 75.6% (n=863) of them were identified as bots during data screening. As a result, a CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart) was added, and after this, no more bots were identified. Manual participant deception occurred throughout the study. Of the 5956 participants (excluding bots) who enrolled in the study, 298 (5%) were identified as false participants. The extent of this decreased from 110 in November 2020, to a negligible level by February 2022 including a number of months with 0. The decline occurred after we added further screening questions such as attention checks, removed the prominence of financial compensation from social media advertising, and added an additional requirement to provide a mobile phone number for identity verification. Conclusions: Data management protocols are necessary to detect automated bots and manual participant deception in remotely conducted trials. Bots and manual deception can be minimized by adding a CAPTCHA, attention checks, a requirement to provide a phone number for identity verification, and not prominently advertising financial compensation on social media. Trial Registration: ISRCTN Number ISRCTN64052601; https://doi.org/10.1186/ISRCTN64052601