AFAnna-Lena Frey

Anna-Lena Frey

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Anna-Lena Frey has a total of 60 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Anhedonia is associated with computational impairments in reward and effort learning in young people with depression symptoms

    by Angad Sahni, Anna-Lena Frey, Ciara McCabe

    Background: Anhedonia and depression symptoms have been linked to potential deficits in reward learning. However, how anhedonia impacts the ability to adjust and learn about the effort required to obtain rewards remains unclear. Methods: We examined young people ( N = 155, 16–25 years) with a range of depression and anhedonia symptoms using a probabilistic instrumental reward and effort learning task. Participants were asked to learn which options to choose to maximize reward or minimize effort for reward. We compared the exerted effort (button pressing speed) for high (puppy images) vs low (dog images) rewards and collected subjective reports of “liking,” “wanting,” and “willingness to exert effort.” Computational models were fit to the learning data and estimated parameter values were correlated with depression and anhedonia symptoms. Results: As depression symptoms and consummatory anhedonia increased, reward liking decreased, and as anticipatory anhedonia increased, liking, wanting, and willingness to exert effort for reward decreased. Participants exerted more effort for high rewards than for low rewards, but anticipatory anhedonia diminished this difference. Higher consummatory anhedonia was associated with poorer reward and effort learning, and with increased temperature parameter values for both learning types, indicating a higher tendency to make exploratory choices. Higher depression symptoms were associated with lower reward learning accuracy. Conclusion: We provide novel evidence that anhedonia is associated with difficulties in modulating effort as a function of reward value and with the underexploitation of low effort and high reward options. We suggest that addressing these impairments could be a novel target for intervention in anhedonic young people.


  • Open Access

    Preferences and Willingness to Pay for Health App Assessments Among Health Care Stakeholders: Discrete Choice Experiment

    by Anna-Lena Frey, Simon Leigh, Carla Toro, Carme Pratdepàdua Bufill, Charles McCay, Tatjana Prenđa Trupec, Giuseppe D'AVenio, Menno Kok, Antanas Montvila, Philipp Goedecker, et al.

    Background: The adoption of high-quality health apps has been slow, despite the myriad benefits associated with their use. This is partly due to concerns regarding the effectiveness, safety, and data privacy of such apps. Quality assessments with robust and transparent criteria can address these concerns and, thereby, encourage the use of high-quality apps. However, a major challenge for such assessments is reaching a scale at which a substantial proportion of the more than 350,000 available health apps can be evaluated. Objective: To support the scaling of health app quality assessments, this study aimed to examine the preferences and willingness to pay for assessments with different value propositions among potential customers. Methods: We conducted 2 discrete choice experiments: one with 41 health app developers and another with 46 health system representatives (from health care institutions, authorities, and insurers) from across Europe. Mixed logit models were applied to examine the impact of assessment attributes on participants’ choices as well as to calculate marginal willingness to pay and predicted assessment uptake. Results: Among health app developers, the attributes with the largest impact on assessment choices were the associated clinical care uptake (integration into clinical guidelines and reimbursement or procurement) and cost (purchase price). Increased willingness to use assessed apps and app store integration of assessment results had a moderate impact on choices, while required developer time investment and time until assessment results become available made the smallest contribution. Among health system representatives, increased willingness of clinicians and patients to use evaluated apps had the greatest impact on assessment choices, followed by cost. Time until assessment result availability and the percentage of peers recommending the assessment made a moderate contribution, while reassessment frequency had the smallest impact on choices. On average, health app developers were willing to pay an additional €9020 (95% CI €4968-€13,072) if an assessment facilitates guideline integration and procurement or reimbursement (at the time of data collection, €1=US $1.11), while health system representatives were, on average, willing to pay €7037 (95% CI €4267-€9806) more if an assessment results in a large, rather than a small, increase in willingness to use the evaluated app. The predicted uptake of assessments that offer the preferred values for all attributes was 88.6% among app developers and 91.1% among health system representatives. Conclusions: These findings indicate that, to maximize uptake and willingness to pay among health app developers, it is advisable for assessments to facilitate or enable clinical guideline integration and reimbursement or procurement for high-scoring apps. Assessment scaling thus requires close collaboration with health authorities, health care institutions, and insurers. Furthermore, if health system organizations are targeted as customers, it is essential to provide evidence for the assessment’s impact on patients’ and clinicians’ willingness to use health apps.


  • Open Access

    Testing and Iterative Improvement of the CEN ISO/TS 82304-2 Health App Quality Assessment: Pilot Interrater Reliability Study

    by Anna-Lena Frey, Diana Matei, Ben Phillips, Adam McCabe, Rachel Fuller, Begoña Laibarra, Laura Alonso, Victor de la Hoz, Carme Pratdepadua Bufill, Berta Llebot Casajuana, et al.

    Background: With the increasing use of health apps and ongoing concerns regarding their safety, effectiveness, and data privacy, numerous health app quality assessment frameworks have emerged. However, assessment initiatives experience difficulties scaling, and there is currently no comprehensive, consistent, internationally recognized assessment framework. Therefore, health apps often need to undergo several quality evaluations to enter different markets, leading to duplication of work. The CEN ISO/TS 82304‑2 health app assessment seeks to address this issue, aiming to provide an internationally accepted quality evaluation through a network of assessment organizations located in different countries. Objective: This study aimed to develop and evolve the draft CEN ISO/TS 82304-2 assessment handbook and developer guidance by testing them across organizations in several countries. Methods: Assessment organizations from 5 countries were engaged to evaluate 24 health apps using the evolving CEN ISO/TS 82304-2 assessment across 3 evaluation rounds. The information submitted by a given health app developer was evaluated by 2 assessment organizations, and interrater reliability was examined. In addition, app developers and assessors were asked to report how much time they spent on information collation or evaluation and to rate the clarity of the developer guidance or assessor handbook, respectively. The collected data were used to iteratively improve the handbook and guidance between rounds. Results: The interrater reliability between assessment organizations improved from round 1 to round 2 and stayed relatively stable between rounds 2 and 3, with 80% (55/69) of assessment questions demonstrating moderate or better (Gwet AC1>0.41) agreement in round 3. The median time required by developers to prepare the assessment information was 8 hours and 59 minutes (IQR 5.7-27.1 hours) in round 3, whereas assessors reported a median evaluation time of 8 hours and 46 minutes (IQR 7.1-11.0 hours). The draft guidance and handbook were generally perceived as clear, with a median round-3 clarity rating of 1.73 (IQR 1.64-1.90) for developers and 1.78 (IQR 1.71-1.89) for assessors (0=“very unclear”, 1=“somewhat unclear”, and 2=“completely clear”). Conclusions: To our knowledge, this is the first study to examine the consistency of health app evaluations across organizations located in different countries. Given that the CEN ISO/TS 82304-2 guidance and handbook are still under development, the interrater reliability findings observed at this early stage are promising, and this study provided valuable information for further refinement of the assessment. This study marks an important first step toward establishing the CEN ISO/TS 82304-2 assessment as a consistent, cross-national health app evaluation. It is envisioned that the assessment will ultimately help avoid duplication of work, prevent inequities by facilitating access to smaller markets for developers, and build trust among users, thereby increasing the adoption of high-quality health apps.


  • Domain coverage and criteria overlap across digital health technology quality assessments: a systematic review

    by Anna-Lena Frey, Ben Phillips, Rebecca Baines, Adam McCabe, Evelyn Elmes, Emily Yeardsley-Pierce, Rachel Wall, Jake Parry, Alice Vose, Jack Hewitt, et al.


  • Open Access

    Testing and Iterative Improvement of the CEN ISO/TS 82304-2 Health App Quality Assessment: Pilot Interrater Reliability Study (Preprint)

    by Anna-Lena Frey, Diana Matei, Ben Phillips, Adam McCabe, Rachel Fuller, Begoña Laibarra, Laura Alonso, Victor de la Hoz, Carme Pratdepadua Bufill, Berta Llebot Casajuana, et al.

    BACKGROUND With the increasing use of health apps and ongoing concerns regarding their safety, effectiveness, and data privacy, numerous health app quality assessment frameworks have emerged. However, assessment initiatives experience difficulties scaling, and there is currently no comprehensive, consistent, internationally recognized assessment framework. Therefore, health apps often need to undergo several quality evaluations to enter different markets, leading to duplication of work. The CEN ISO/TS 82304‑2 health app assessment seeks to address this issue, aiming to provide an internationally accepted quality evaluation through a network of assessment organizations located in different countries. OBJECTIVE This study aimed to develop and evolve the draft CEN ISO/TS 82304-2 assessment handbook and developer guidance by testing them across organizations in several countries. METHODS Assessment organizations from 5 countries were engaged to evaluate 24 health apps using the evolving CEN ISO/TS 82304-2 assessment across 3 evaluation rounds. The information submitted by a given health app developer was evaluated by 2 assessment organizations, and interrater reliability was examined. In addition, app developers and assessors were asked to report how much time they spent on information collation or evaluation and to rate the clarity of the developer guidance or assessor handbook, respectively. The collected data were used to iteratively improve the handbook and guidance between rounds. RESULTS The interrater reliability between assessment organizations improved from round 1 to round 2 and stayed relatively stable between rounds 2 and 3, with 80% (55/69) of assessment questions demonstrating moderate or better (Gwet AC1>0.41) agreement in round 3. The median time required by developers to prepare the assessment information was 8 hours and 59 minutes (IQR 5.7-27.1 hours) in round 3, whereas assessors reported a median evaluation time of 8 hours and 46 minutes (IQR 7.1-11.0 hours). The draft guidance and handbook were generally perceived as clear, with a median round-3 clarity rating of 1.73 (IQR 1.64-1.90) for developers and 1.78 (IQR 1.71-1.89) for assessors (0=“very unclear”, 1=“somewhat unclear”, and 2=“completely clear”). CONCLUSIONS To our knowledge, this is the first study to examine the consistency of health app evaluations across organizations located in different countries. Given that the CEN ISO/TS 82304-2 guidance and handbook are still under development, the interrater reliability findings observed at this early stage are promising, and this study provided valuable information for further refinement of the assessment. This study marks an important first step toward establishing the CEN ISO/TS 82304-2 assessment as a consistent, cross-national health app evaluation. It is envisioned that the assessment will ultimately help avoid duplication of work, prevent inequities by facilitating access to smaller markets for developers, and build trust among users, thereby increasing the adoption of high-quality health apps.


  • Open Access

    Preferences and Willingness to Pay for Health App Assessments Among Health Care Stakeholders: Discrete Choice Experiment (Preprint)

    by Anna-Lena Frey, Simon Leigh, Carla Toro, Carme Pratdepàdua Bufill, Charles McCay, Tatjana Prenđa Trupec, Giuseppe D'Avenio, Menno Kok, Antanas Montvila, Philipp Goedecker, et al.

    BACKGROUND The adoption of high-quality health apps has been slow, despite the myriad benefits associated with their use. This is partly due to concerns regarding the effectiveness, safety, and data privacy of such apps. Quality assessments with robust and transparent criteria can address these concerns and, thereby, encourage the use of high-quality apps. However, a major challenge for such assessments is reaching a scale at which a substantial proportion of the more than 350,000 available health apps can be evaluated. OBJECTIVE To support the scaling of health app quality assessments, this study aimed to examine the preferences and willingness to pay for assessments with different value propositions among potential customers. METHODS We conducted 2 discrete choice experiments: one with 41 health app developers and another with 46 health system representatives (from health care institutions, authorities, and insurers) from across Europe. Mixed logit models were applied to examine the impact of assessment attributes on participants’ choices as well as to calculate marginal willingness to pay and predicted assessment uptake. RESULTS Among health app developers, the attributes with the largest impact on assessment choices were the associated clinical care uptake (integration into clinical guidelines and reimbursement or procurement) and cost (purchase price). Increased willingness to use assessed apps and app store integration of assessment results had a moderate impact on choices, while required developer time investment and time until assessment results become available made the smallest contribution. Among health system representatives, increased willingness of clinicians and patients to use evaluated apps had the greatest impact on assessment choices, followed by cost. Time until assessment result availability and the percentage of peers recommending the assessment made a moderate contribution, while reassessment frequency had the smallest impact on choices. On average, health app developers were willing to pay an additional €9020 (95% CI €4968-€13,072) if an assessment facilitates guideline integration and procurement or reimbursement (at the time of data collection, €1=US $1.11), while health system representatives were, on average, willing to pay €7037 (95% CI €4267-€9806) more if an assessment results in a large, rather than a small, increase in willingness to use the evaluated app. The predicted uptake of assessments that offer the preferred values for all attributes was 88.6% among app developers and 91.1% among health system representatives. CONCLUSIONS These findings indicate that, to maximize uptake and willingness to pay among health app developers, it is advisable for assessments to facilitate or enable clinical guideline integration and reimbursement or procurement for high-scoring apps. Assessment scaling thus requires close collaboration with health authorities, health care institutions, and insurers. Furthermore, if health system organizations are targeted as customers, it is essential to provide evidence for the assessment’s impact on patients’ and clinicians’ willingness to use health apps.


  • Open Access

    Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study

    by Maciej Hyzy, Raymond Bond, Maurice Mulvenna, Lu Bai, Alan Dix, Robert Daly, Anna-Lena Frey, Simon Leigh

    Background: There are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards. Objective: The objective of this paper was to examine and compare the compliance of DHIs with best practice standards and adherence to user experience (UX), professional and clinical assurance (PCA), and data privacy (DP). Methods: We collected assessment data from 1574 DHIs using the Organisation for the Review of Care and Health Apps Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned areas (ie, UX, PCA, and DP). These 3 OBR scores are combined to make up the overall ORCHA score (a proxy for quality). Inferential statistics, probability distributions, Kruskal-Wallis, Wilcoxon rank sum test, Cliff delta, and Dunn tests were used to conduct the data analysis. Results: We found that 57.3% (902/1574) of the DHIs had an Organisation for the Review of Care and Health Apps (ORCHA) score below the threshold of 65. The overall median OBR score (ORCHA score) for all DHIs was 61.5 (IQR 51.0-73.0) out of 100. A total of 46.2% (12/26) of DHI’s health care domains had a median equal to or above the ORCHA threshold score of 65. For the 3 assessment areas (UX, DP, and PCA), DHIs scored the highest for the UX assessment 75.2 (IQR 70.0-79.6), followed by DP 65.1 (IQR 55.0-73.4) and PCA 49.6 (IQR 31.9-76.1). UX scores had the least variance (SD 13.9), while PCA scores had the most (SD 24.8). Respiratory and urology DHIs were consistently highly ranked in the National Institute for Health and Care Excellence Evidence Standards Framework tiers B and C based on their ORCHA score. Conclusions: There is a high level of variability in the ORCHA scores of DHIs across different health care domains. This suggests that there is an urgent need to improve compliance with best practices in some health care areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different health care domains. More investment to support the development of higher-quality DHIs in areas such as ophthalmology, allergy, women’s health, sexual health, and dental care may be needed.


  • Open Access

    Association Between the Characteristics of mHealth Apps and User Input During Development and Testing: Secondary Analysis of App Assessment Data

    by Anna-Lena Frey, Rebecca Baines, Sophie Hunt, Rachael Kent, Tim Andrews, Simon Leigh

    Background: User involvement is increasingly acknowledged as a central part of health care innovation. However, meaningful user involvement during the development and testing of mobile health apps is often not fully realized. Objective: This study aims to examine in which areas user input is most prevalent and whether there is an association between user inclusion and compliance with best practices for mobile health apps. Methods: A secondary analysis was conducted on an assessment data set of 1595 health apps. The data set contained information on whether the apps had been developed or tested with user input and whether they followed best practices across several domains. Background information was also available regarding the apps’ country of origin, targeted condition areas, subjective user ratings, download numbers, and risk (as per the National Institute for Health and Care Excellence Evidence Standards Framework [ESF]). Descriptive statistics, Mann-Whitney U tests, and Pearson chi-square analyses were applied to the data. Results: User involvement was reported by 8.71% (139/1595) of apps for only the development phase, by 33.67% (537/1595) of apps for only the testing phase, by 21.88% (349/1595) of apps for both phases, and by 35.74% (570/1595) of apps for neither phase. The highest percentage of health apps with reported user input during development was observed in Denmark (19/24, 79%); in the condition areas of diabetes (38/79, 48%), cardiology (15/32, 47%), pain management (20/43, 47%), and oncology (25/54, 46%); and for high app risk (ESF tier 3a; 105/263, 39.9%). The highest percentage of health apps with reported user input during testing was observed in Belgium (10/11, 91%), Sweden (29/34, 85%), and France (13/16, 81%); in the condition areas of neurodiversity (42/52, 81%), respiratory health (58/76, 76%), cardiology (23/32, 72%), and diabetes (56/79, 71%); and for high app risk (ESF tier 3a; 176/263, 66.9%). Notably, apps that reported seeking user input during testing demonstrated significantly more downloads than those that did not (P=.008), and user inclusion was associated with better compliance with best practices in clinical assurance, data privacy, risk management, and user experience. Conclusions: The countries and condition areas in which the highest percentage of health apps with user involvement were observed tended to be those with higher digital maturity in health care and more funding availability, respectively. This suggests that there may be a trade-off between developers’ willingness or ability to involve users and the need to meet challenges arising from infrastructure limitations and financial constraints. Moreover, the finding of a positive association between user inclusion and compliance with best practices indicates that, where no other guidance is available, users may benefit from prioritizing health apps developed with user input as the latter may be a proxy for broader app quality.


  • Open Access

    Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study (Preprint)

    by Maciej Hyzy, Raymond Bond, Maurice Mulvenna, Lu Bai, Alan Dix, Robert Daly, Anna-Lena Frey, Simon Leigh

    BACKGROUND There are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards. OBJECTIVE The objective of this paper was to examine and compare the compliance of DHIs with best practice standards and adherence to user experience (UX), professional and clinical assurance (PCA), and data privacy (DP). METHODS We collected assessment data from 1574 DHIs using the Organisation for the Review of Care and Health Apps Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned areas (ie, UX, PCA, and DP). These 3 OBR scores are combined to make up the overall ORCHA score (a proxy for quality). Inferential statistics, probability distributions, Kruskal-Wallis, Wilcoxon rank sum test, Cliff delta, and Dunn tests were used to conduct the data analysis. RESULTS We found that 57.3% (902/1574) of the DHIs had an Organisation for the Review of Care and Health Apps (ORCHA) score below the threshold of 65. The overall median OBR score (ORCHA score) for all DHIs was 61.5 (IQR 51.0-73.0) out of 100. A total of 46.2% (12/26) of DHI’s health care domains had a median equal to or above the ORCHA threshold score of 65. For the 3 assessment areas (UX, DP, and PCA), DHIs scored the highest for the UX assessment 75.2 (IQR 70.0-79.6), followed by DP 65.1 (IQR 55.0-73.4) and PCA 49.6 (IQR 31.9-76.1). UX scores had the least variance (SD 13.9), while PCA scores had the most (SD 24.8). Respiratory and urology DHIs were consistently highly ranked in the National Institute for Health and Care Excellence Evidence Standards Framework tiers B and C based on their ORCHA score. CONCLUSIONS There is a high level of variability in the ORCHA scores of DHIs across different health care domains. This suggests that there is an urgent need to improve compliance with best practices in some health care areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different health care domains. More investment to support the development of higher-quality DHIs in areas such as ophthalmology, allergy, women’s health, sexual health, and dental care may be needed.