Caoimhe Cawley

Background: Unprecedented lockdown measures have been introduced in countries across the world to mitigate the spread and consequences of COVID-19. While attention has focused on the effects of these measures on epidemiological indicators relating directly to the infection, there is increased recognition of their broader health implications. However, assessing these implications in real time is a challenge, due to limitations of existing syndromic surveillance data and tools.Objective: To explore the added value of mobile phone app-based symptom assessment tools as real time health insight providers to inform public health policy makers.Methods: A comparative and descriptive analysis of the proportion of all self-reported symptoms entered by users during an Ada assessment in Germany and the United Kingdom (UK) was conducted between two periods: before and after the implementation of “Phase One” COVID-19 measures. Additional analyses were performed to explore the association between symptom trends and seasonality, and symptom trends and weather. Differences in the proportion of unique symptoms between the periods were analysed using Pearson’s Chi-squared test and reported as Log2 Fold Changes (Log2 FC).Results: Between 48,300-54,900 symptomatic users reported 140,500-170,400 symptoms during the Baseline and Measures periods in Germany. Between 34,200-37,400 symptomatic users in the UK reported 112,100-131,900 symptoms during the Baseline and Measures periods. The majority of symptomatic users were female (Germany 68,600/103,200, 66.52%; UK 51,200/71,600, 72.74%). The majority (Germany 68,500/100,000, 68.45%; UK 50,900/68,800, 73.91%) were aged between 10 and 29 years, and about a quarter (Germany 26,200/100,000, 26.15%; UK 14,900/68,800, 21.65%) were between 30-59 years. 103 symptoms were reported either more or less frequently (with statistically significant differences) during the Measures as compared to the Baseline period, and 34 of these were found in both countries. The following mental health symptoms (Log2 FC, P-value) were reported less often during the Measures period: inability to manage constant stress and demands at work (−1.07, P<.001), memory difficulty (−0.56, P<.001), depressed mood (−0.42, P<.001), and impaired concentration (−0.46, P<.001). Diminished sense of taste (2.26, P<.001) and hyposmia (2.20, P<.001) were reported more frequently during the Measures period. None of the 34 symptoms were found to be different between the same dates in 2019. Fourteen of the 34 symptoms had statistically significant associations with weather variables.Conclusions: Symptom assessment apps have an important role to play in facilitating improved understanding of the implications of public health policies such as COVID-19 lockdown measures. Not only do they provide the means to complement and cross-validate hypotheses based on data collected through more traditional channels, they can also generate novel insights through a real-time syndromic surveillance system.

Objectives: To compare breadth of condition coverage, accuracy of suggested conditions and appropriateness of urgency advice of 8 popular symptom assessment apps with each other and with 7 General Practitioners. Design: Clinical vignettes study. Setting: 200 clinical vignettes representing real-world scenarios in primary care. Intervention/comparator: Condition coverage, suggested condition accuracy, and urgency advice performance was measured against the vignettes’ gold-standard diagnoses and triage level. Primary outcome measures: Outcomes included (i) proportion of conditions “covered” by an app, i.e. not excluded because the patient was too young/old, pregnant, or comorbid, (ii) proportion of vignettes in which the correct primary diagnosis was amongst the top 3 conditions suggested, and, (iii) proportion of “safe” urgency level advice (i.e. at gold standard level, more conservative, or no more than one level less conservative). Results: Condition-suggestion coverage was highly variable, with some apps not offering a suggestion for many users: in alphabetical order, Ada: 99.0%; Babylon: 51.5%; Buoy: 88.5%; K Health: 74.5%; Mediktor: 80.5%; Symptomate: 61.5%; Your.MD: 64.5%. The top-3 suggestion accuracy (M3) of GPs was on average 82.i±5.2%. For the apps it was - Ada: 70.5%; Babylon: 32.0%; Buoy: 43.0%; K Health: 36.0%; Mediktor: 36.0%; Symptomate: 27.5%; WebMD: 35.5%; Your.MD: 23.5%. Some apps exclude certain user groups (e.g. younger users) or certain conditions - for these apps condition-suggestion performance is generally greater with exclusion of these vignettes. For safe urgency advice, tested GPs had an average of 97.0±2.5%. For the vignettes with advice provided, only three apps had safety performance within 1 S.D. of the GPs (mean) - Ada: 97.0%; Babylon: 95.i%; Symptomate: 97.8%. One app had a safety performance within 2 S.D.s of GPs - Your.MD: 92.6%. Three apps had a safety performance outside 2 S.D.s of GPs - Buoy: 80.0% ( p p p =1.3×10 ^-3 ). Conclusions: The utility of digital symptom assessment apps relies upon coverage, accuracy, and safety. While no digital tool outperformed GPs, some came close, and the nature of iterative improvements to software offers scalable improvements to care.