Julien Meyer
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Julien Meyer has a total of 27 co-authors. Here are the top co-authors by number of shared publications:
- Pria Nippak
School of Health Services Management , Ted Rogers School of Management , Toronto Metropolitan University , Toronto , Canada , ryerson.ca
5 shared publications David Remisch
School of Health Services Management, Ted Rogers School of Management , Toronto, Ontario, Canada
3 shared publicationsPria Nippak
School of Health Services Management, Ted Rogers School of Management , Toronto, Ontario, Canada
3 shared publicationsApril Khademi
Department of Electrical, Computer, and Biomedical Engineering, Ryerson University , Toronto, Ontario, Canada
2 shared publicationsBernard Têtu
Départment de biologie médicale, Université Laval , Québec City, Quebec, Canada
2 shared publications- Deanna E Wiebe
Department of Health Services Management Ryerson University Toronto, ON Canada
2 shared publications
Latest Publications
- Open Access
- 21 October 2022
- Published by Wolters Kluwer Health in Medicine
- Vol. 101 (42) , e31047
Opioid prescribing practices in trauma patients at discharge: An exploratory retrospective chart analysis
This study examined the opioid prescribing patterns at discharge in the trauma center of a major Canadian hospital and compared them to the guidelines provided by the Illinois surgical quality improvement collaborative (ISQIC), a framework that has been recognized as being associated with reduced risk. This was a retrospective chart review of patient data from the trauma registry between January 1, 2018, and October 31, 2019. A total of 268 discharge charts of naïve opioid patients were included in the analysis. A Morphine Milligram Equivalents per day (MME/day) was computed for each patient who was prescribed opioids and compared with standard practice guidelines. About 75% of patients were prescribed opioids. More males (75%) than females (25%) were prescribed opioids to patients below 65 years old (91%). Best practice guidelines were followed in most cases. Only 16.6% of patients were prescribed over 50 mg MME/day, the majority (80.9%) were prescribed opioids for =7 days. Only 7.5% were prescribed extended-release opioids and none were strong like fentanyl. Patients received a multimodal approach with alternatives to opioids in 88.9% of cases and 82.9% had a plan for opioid discontinuation. However, only 23.6% received an acute pain service referral. The majority of the prescriptions provided adhered to the best practice guidelines outlined by the ISQIC framework. These results are encouraging with respect to the feasibility of implementing opioid prescription guidelines effectively. However, routine monitoring is necessary to ensure that adherence is maintained.
- 21 October 2022
- Published by Wolters Kluwer Health in Medicine
- Vol. 101 (42) , e31047
Open AccessOpioid prescribing practices in trauma patients at discharge: An exploratory retrospective chart analysis
This study examined the opioid prescribing patterns at discharge in the trauma center of a major Canadian hospital and compared them to the guidelines provided by the Illinois surgical quality improvement collaborative (ISQIC), a framework that has been recognized as being associated with reduced risk. This was a retrospective chart review of patient data from the trauma registry between January 1, 2018, and October 31, 2019. A total of 268 discharge charts of naïve opioid patients were included in the analysis. A Morphine Milligram Equivalents per day (MME/day) was computed for each patient who was prescribed opioids and compared with standard practice guidelines. About 75% of patients were prescribed opioids. More males (75%) than females (25%) were prescribed opioids to patients below 65 years old (91%). Best practice guidelines were followed in most cases. Only 16.6% of patients were prescribed over 50 mg MME/day, the majority (80.9%) were prescribed opioids for =7 days. Only 7.5% were prescribed extended-release opioids and none were strong like fentanyl. Patients received a multimodal approach with alternatives to opioids in 88.9% of cases and 82.9% had a plan for opioid discontinuation. However, only 23.6% received an acute pain service referral. The majority of the prescriptions provided adhered to the best practice guidelines outlined by the ISQIC framework. These results are encouraging with respect to the feasibility of implementing opioid prescription guidelines effectively. However, routine monitoring is necessary to ensure that adherence is maintained.
- 26 July 2022
- Published by Taylor & Francis in Journal of American College Health
- Vol. 72 (7) , 1988-1992
Mental health support for Canadian postsecondary students during COVID-19 pandemic: An environmental scan
This study explored how Canada’s postsecondary institutions have informed students about campus mental health (MH) services via websites and social media during the COVID-19 pandemic.We conducted a...
- 26 July 2022
- Published by Taylor & Francis in Journal of American College Health
- Vol. 72 (7) , 1988-1992
Mental health support for Canadian postsecondary students during COVID-19 pandemic: An environmental scan
This study explored how Canada’s postsecondary institutions have informed students about campus mental health (MH) services via websites and social media during the COVID-19 pandemic.We conducted a...
- 25 June 2022
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 29 (10) , 1688-1695
Impact of artificial intelligence on pathologists’ decisions: an experiment
The accuracy of artificial intelligence (AI) in medicine and in pathology in particular has made major progress but little is known on how much these algorithms will influence pathologists’ decisions in practice. The objective of this paper is to determine the reliance of pathologists on AI and to investigate whether providing information on AI impacts this reliance. The experiment using an online survey design. Under 3 conditions, 116 pathologists and pathology students were tasked with assessing the Gleason grade for a series of 12 prostate biopsies: (1) without AI recommendations, (2) with AI recommendations, and (3) with AI recommendations accompanied by information about the algorithm itself, specifically algorithm accuracy rate and algorithm decision-making process. Participant responses were significantly more accurate with the AI decision aids than without (92% vs 87%, odds ratio 13.30, P < .01). Unexpectedly, the provision of information on the algorithm made no significant difference compared to AI without information. The reliance on AI correlated with general beliefs on AI’s usefulness but not with particular assessments of the AI tool offered. Decisions were made faster when AI was provided. These results suggest that pathologists are willing to rely on AI regardless of accuracy or explanations. Generalization beyond the specific tasks and explanations provided will require further studies. This study suggests that the factors that influence the reliance on AI differ in practice from beliefs expressed by clinicians in surveys. Implementation of AI in prospective settings should take individual behaviors into account.
- 25 June 2022
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 29 (10) , 1688-1695
Impact of artificial intelligence on pathologists’ decisions: an experiment
The accuracy of artificial intelligence (AI) in medicine and in pathology in particular has made major progress but little is known on how much these algorithms will influence pathologists’ decisions in practice. The objective of this paper is to determine the reliance of pathologists on AI and to investigate whether providing information on AI impacts this reliance. The experiment using an online survey design. Under 3 conditions, 116 pathologists and pathology students were tasked with assessing the Gleason grade for a series of 12 prostate biopsies: (1) without AI recommendations, (2) with AI recommendations, and (3) with AI recommendations accompanied by information about the algorithm itself, specifically algorithm accuracy rate and algorithm decision-making process. Participant responses were significantly more accurate with the AI decision aids than without (92% vs 87%, odds ratio 13.30, P < .01). Unexpectedly, the provision of information on the algorithm made no significant difference compared to AI without information. The reliance on AI correlated with general beliefs on AI’s usefulness but not with particular assessments of the AI tool offered. Decisions were made faster when AI was provided. These results suggest that pathologists are willing to rely on AI regardless of accuracy or explanations. Generalization beyond the specific tasks and explanations provided will require further studies. This study suggests that the factors that influence the reliance on AI differ in practice from beliefs expressed by clinicians in surveys. Implementation of AI in prospective settings should take individual behaviors into account.
- Open Access
- 1 April 2022
- Published by JMIR Publications Inc. in JMIR Human Factors
- Vol. 9 (2) , e33493
Smartphone Alcohol Use Disorder Recovery Apps: Cross-sectional Survey of Behavioral Intention to Use
Background: Alcohol use disorder (AUD) carries a huge health and economic cost to society. Effective interventions exist but numerous challenges limit their adoption, especially in a pandemic context. AUD recovery apps (AUDRA) have emerged as a potential complement to in-person interventions. They are easy to access and show promising results in terms of efficacy. However, they rely on individual adoption decisions and remain underused. Objective: The aim of this survey study is to explore the beliefs that determine the intention to use AUDRA. Methods: We conducted a cross-sectional survey study of people with AUD. We used the Unified Theory of Acceptance and Use of Technology, which predicts use and behavioral intention to use based on performance expectancy, effort expectancy, social influence, and facilitating conditions. Participants were recruited directly from 2 sources; first, respondents at addiction treatment facilities in Ontario, Canada, were contacted in person, and they filled a paper form; second, members from AUD recovery support groups on social media were contacted and invited to fill an internet-based survey. The survey was conducted between October 2019 and June 2020. Results: The final sample comprised 159 participants (124 involved in the web-based survey and 35 in the paper-based survey) self-identifying somewhat or very much with AUD. Most participants (n=136, 85.5%) were aware of AUDRA and those participants scored higher on performance expectancy, effort expectancy, and social influence. Overall, the model explains 35.4% of the variance in the behavioral intention to use AUDRA and 11.1% of the variance in use. Social influence (P=.31), especially for women (P=.23), and effort expectancy (P=.25) were key antecedents of behavioral intention. Facilitating conditions were not significant overall but were moderated by age (P=.23), suggesting that it matters for older participants. Performance expectancy did not predict behavioral intention, which is unlike many other technologies but confirms other findings associated with mobile health (mHealth). Open-ended questions suggest that privacy concerns may significantly influence the use of AUDRA. Conclusions: This study suggests that unlike many other technologies, the adoption of AUDRA is not mainly determined by utilitarian factors such as performance expectancy. Rather, effort expectancy and social influence play a key role in determining the intention to use AUDRA.
- 1 April 2022
- Published by JMIR Publications Inc. in JMIR Human Factors
- Vol. 9 (2) , e33493
Open AccessSmartphone Alcohol Use Disorder Recovery Apps: Cross-sectional Survey of Behavioral Intention to Use
Background: Alcohol use disorder (AUD) carries a huge health and economic cost to society. Effective interventions exist but numerous challenges limit their adoption, especially in a pandemic context. AUD recovery apps (AUDRA) have emerged as a potential complement to in-person interventions. They are easy to access and show promising results in terms of efficacy. However, they rely on individual adoption decisions and remain underused. Objective: The aim of this survey study is to explore the beliefs that determine the intention to use AUDRA. Methods: We conducted a cross-sectional survey study of people with AUD. We used the Unified Theory of Acceptance and Use of Technology, which predicts use and behavioral intention to use based on performance expectancy, effort expectancy, social influence, and facilitating conditions. Participants were recruited directly from 2 sources; first, respondents at addiction treatment facilities in Ontario, Canada, were contacted in person, and they filled a paper form; second, members from AUD recovery support groups on social media were contacted and invited to fill an internet-based survey. The survey was conducted between October 2019 and June 2020. Results: The final sample comprised 159 participants (124 involved in the web-based survey and 35 in the paper-based survey) self-identifying somewhat or very much with AUD. Most participants (n=136, 85.5%) were aware of AUDRA and those participants scored higher on performance expectancy, effort expectancy, and social influence. Overall, the model explains 35.4% of the variance in the behavioral intention to use AUDRA and 11.1% of the variance in use. Social influence (P=.31), especially for women (P=.23), and effort expectancy (P=.25) were key antecedents of behavioral intention. Facilitating conditions were not significant overall but were moderated by age (P=.23), suggesting that it matters for older participants. Performance expectancy did not predict behavioral intention, which is unlike many other technologies but confirms other findings associated with mobile health (mHealth). Open-ended questions suggest that privacy concerns may significantly influence the use of AUDRA. Conclusions: This study suggests that unlike many other technologies, the adoption of AUDRA is not mainly determined by utilitarian factors such as performance expectancy. Rather, effort expectancy and social influence play a key role in determining the intention to use AUDRA.
- Open Access
- 14 December 2021
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 5 (12) , e17062
User Reviews of Depression App Features: Sentiment Analysis
Background: Mental health in general, and depression in particular, remain undertreated conditions. Mobile health (mHealth) apps offer tremendous potential to overcome the barriers to accessing mental health care and millions of depression apps have been installed and used. However, little is known about the effect of these apps on a potentially vulnerable user population and the emotional reactions that they generate, even though emotions are a key component of mental health. App reviews, spontaneously posted by the users on app stores, offer up-to-date insights into the experiences and emotions of this population and are increasingly decisive in influencing mHealth app adoption. Objective: This study aims to investigate the emotional reactions of depression app users to different app features by systematically analyzing the sentiments expressed in app reviews. Methods: We extracted 3261 user reviews of depression apps. The 61 corresponding apps were categorized by the features they offered (psychoeducation, medical assessment, therapeutic treatment, supportive resources, and entertainment). We then produced word clouds by features and analyzed the reviews using the Linguistic Inquiry Word Count 2015 (Pennebaker Conglomerates, Inc), a lexicon-based natural language analytical tool that analyzes the lexicons used and the valence of a text in 4 dimensions (authenticity, clout, analytic, and tone). We compared the language patterns associated with the different features of the underlying apps. Results: The analysis highlighted significant differences in the sentiments expressed for the different features offered. Psychoeducation apps exhibited more clout but less authenticity (ie, personal disclosure). Medical assessment apps stood out for the strong negative emotions and the relatively negative ratings that they generated. Therapeutic treatment app features generated more positive emotions, even though user feedback tended to be less authentic but more analytical (ie, more factual). Supportive resources (connecting users to physical services and people) and entertainment apps also generated fewer negative emotions and less anxiety. Conclusions: Developers should be careful in selecting the features they offer in their depression apps. Medical assessment features may be riskier as users receive potentially disturbing feedback on their condition and may react with strong negative emotions. In contrast, offering information, contacts, or even games may be safer starting points to engage people with depression at a distance. We highlight the necessity to differentiate how mHealth apps are assessed and vetted based on the features they offer. Methodologically, this study points to novel ways to investigate the impact of mHealth apps and app features on people with mental health issues. mHealth apps exist in a rapidly changing ecosystem that is driven by user satisfaction and adoption decisions. As such, user perceptions are essential and must be monitored to ensure adoption and avoid harm to a fragile population that may not benefit from traditional health care resources.
- 14 December 2021
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 5 (12) , e17062
Open AccessUser Reviews of Depression App Features: Sentiment Analysis
Background: Mental health in general, and depression in particular, remain undertreated conditions. Mobile health (mHealth) apps offer tremendous potential to overcome the barriers to accessing mental health care and millions of depression apps have been installed and used. However, little is known about the effect of these apps on a potentially vulnerable user population and the emotional reactions that they generate, even though emotions are a key component of mental health. App reviews, spontaneously posted by the users on app stores, offer up-to-date insights into the experiences and emotions of this population and are increasingly decisive in influencing mHealth app adoption. Objective: This study aims to investigate the emotional reactions of depression app users to different app features by systematically analyzing the sentiments expressed in app reviews. Methods: We extracted 3261 user reviews of depression apps. The 61 corresponding apps were categorized by the features they offered (psychoeducation, medical assessment, therapeutic treatment, supportive resources, and entertainment). We then produced word clouds by features and analyzed the reviews using the Linguistic Inquiry Word Count 2015 (Pennebaker Conglomerates, Inc), a lexicon-based natural language analytical tool that analyzes the lexicons used and the valence of a text in 4 dimensions (authenticity, clout, analytic, and tone). We compared the language patterns associated with the different features of the underlying apps. Results: The analysis highlighted significant differences in the sentiments expressed for the different features offered. Psychoeducation apps exhibited more clout but less authenticity (ie, personal disclosure). Medical assessment apps stood out for the strong negative emotions and the relatively negative ratings that they generated. Therapeutic treatment app features generated more positive emotions, even though user feedback tended to be less authentic but more analytical (ie, more factual). Supportive resources (connecting users to physical services and people) and entertainment apps also generated fewer negative emotions and less anxiety. Conclusions: Developers should be careful in selecting the features they offer in their depression apps. Medical assessment features may be riskier as users receive potentially disturbing feedback on their condition and may react with strong negative emotions. In contrast, offering information, contacts, or even games may be safer starting points to engage people with depression at a distance. We highlight the necessity to differentiate how mHealth apps are assessed and vetted based on the features they offer. Methodologically, this study points to novel ways to investigate the impact of mHealth apps and app features on people with mental health issues. mHealth apps exist in a rapidly changing ecosystem that is driven by user satisfaction and adoption decisions. As such, user perceptions are essential and must be monitored to ensure adoption and avoid harm to a fragile population that may not benefit from traditional health care resources.
- Open Access
- 1 December 2021
- Published by JMIR Publications Inc. in JMIR Mental Health
- Vol. 8 (12) , e29243
Evaluation of an Online System for Routine Outcome Monitoring: Cross-sectional Survey Study
Background: The use of routine outcome monitoring (ROM) in the treatment of mental health has emerged as a method of improving psychotherapy treatment outcomes. Despite this, very few clinicians regularly use ROM in clinical practice. Online ROM has been suggested as a solution to increase adoption. Objective: The aim of this study is to identify the influence of moving ROM online on client completion rates of self-reported outcome measures and to identify implementation and utilization barriers to online ROM by assessing clinicians’ views on their experience using the online system over previous paper-based methods. Methods: Client completion rates of self-reported outcome measures were compared pre- and postimplementation of an online system of ROM. In addition, a survey questionnaire was administered to 324 mental health service providers regarding their perception of the benefits with an online system of ROM. Results: Client completion rates of self-reported measures increased from 15.62% (427/2734) to 53.98% (1267/2347) after they were moved online. Furthermore, 57% (56/98) of service providers found the new system less time-consuming than the previous paper-based ROM, and 64% (63/98) found that it helped monitor clients. However, the perceived value of the system remains in doubt as only 23% (23/98) found it helped them identify clients at risk for treatment failure, and only 18% (18/98) found it strengthened the therapeutic alliance. Conclusions: Although the current study suggests mixed results regarding service providers’ views on their experience using an online system for ROM, it has identified barriers and challenges that are actionable for improvement.
- 1 December 2021
- Published by JMIR Publications Inc. in JMIR Mental Health
- Vol. 8 (12) , e29243
Open AccessEvaluation of an Online System for Routine Outcome Monitoring: Cross-sectional Survey Study
Background: The use of routine outcome monitoring (ROM) in the treatment of mental health has emerged as a method of improving psychotherapy treatment outcomes. Despite this, very few clinicians regularly use ROM in clinical practice. Online ROM has been suggested as a solution to increase adoption. Objective: The aim of this study is to identify the influence of moving ROM online on client completion rates of self-reported outcome measures and to identify implementation and utilization barriers to online ROM by assessing clinicians’ views on their experience using the online system over previous paper-based methods. Methods: Client completion rates of self-reported outcome measures were compared pre- and postimplementation of an online system of ROM. In addition, a survey questionnaire was administered to 324 mental health service providers regarding their perception of the benefits with an online system of ROM. Results: Client completion rates of self-reported measures increased from 15.62% (427/2734) to 53.98% (1267/2347) after they were moved online. Furthermore, 57% (56/98) of service providers found the new system less time-consuming than the previous paper-based ROM, and 64% (63/98) found that it helped monitor clients. However, the perceived value of the system remains in doubt as only 23% (23/98) found it helped them identify clients at risk for treatment failure, and only 18% (18/98) found it strengthened the therapeutic alliance. Conclusions: Although the current study suggests mixed results regarding service providers’ views on their experience using an online system for ROM, it has identified barriers and challenges that are actionable for improvement.
- Open Access
- 9 September 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Smartphone Alcohol Use Disorder Recovery Apps: Cross-sectional Survey of Behavioral Intention to Use (Preprint)
BACKGROUND Alcohol use disorder (AUD) carries a huge health and economic cost to society. Effective interventions exist but numerous challenges limit their adoption, especially in a pandemic context. AUD recovery apps (AUDRA) have emerged as a potential complement to in-person interventions. They are easy to access and show promising results in terms of efficacy. However, they rely on individual adoption decisions and remain underused. OBJECTIVE The aim of this survey study is to explore the beliefs that determine the intention to use AUDRA. METHODS We conducted a cross-sectional survey study of people with AUD. We used the Unified Theory of Acceptance and Use of Technology, which predicts use and behavioral intention to use based on performance expectancy, effort expectancy, social influence, and facilitating conditions. Participants were recruited directly from 2 sources; first, respondents at addiction treatment facilities in Ontario, Canada, were contacted in person, and they filled a paper form; second, members from AUD recovery support groups on social media were contacted and invited to fill an internet-based survey. The survey was conducted between October 2019 and June 2020. RESULTS The final sample comprised 159 participants (124 involved in the web-based survey and 35 in the paper-based survey) self-identifying somewhat or very much with AUD. Most participants (n=136, 85.5%) were aware of AUDRA and those participants scored higher on performance expectancy, effort expectancy, and social influence. Overall, the model explains 35.4% of the variance in the behavioral intention to use AUDRA and 11.1% of the variance in use. Social influence (P=.31), especially for women (P=.23), and effort expectancy (P=.25) were key antecedents of behavioral intention. Facilitating conditions were not significant overall but were moderated by age (P=.23), suggesting that it matters for older participants. Performance expectancy did not predict behavioral intention, which is unlike many other technologies but confirms other findings associated with mobile health (mHealth). Open-ended questions suggest that privacy concerns may significantly influence the use of AUDRA. CONCLUSIONS This study suggests that unlike many other technologies, the adoption of AUDRA is not mainly determined by utilitarian factors such as performance expectancy. Rather, effort expectancy and social influence play a key role in determining the intention to use AUDRA.
- 9 September 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessSmartphone Alcohol Use Disorder Recovery Apps: Cross-sectional Survey of Behavioral Intention to Use (Preprint)
BACKGROUND Alcohol use disorder (AUD) carries a huge health and economic cost to society. Effective interventions exist but numerous challenges limit their adoption, especially in a pandemic context. AUD recovery apps (AUDRA) have emerged as a potential complement to in-person interventions. They are easy to access and show promising results in terms of efficacy. However, they rely on individual adoption decisions and remain underused. OBJECTIVE The aim of this survey study is to explore the beliefs that determine the intention to use AUDRA. METHODS We conducted a cross-sectional survey study of people with AUD. We used the Unified Theory of Acceptance and Use of Technology, which predicts use and behavioral intention to use based on performance expectancy, effort expectancy, social influence, and facilitating conditions. Participants were recruited directly from 2 sources; first, respondents at addiction treatment facilities in Ontario, Canada, were contacted in person, and they filled a paper form; second, members from AUD recovery support groups on social media were contacted and invited to fill an internet-based survey. The survey was conducted between October 2019 and June 2020. RESULTS The final sample comprised 159 participants (124 involved in the web-based survey and 35 in the paper-based survey) self-identifying somewhat or very much with AUD. Most participants (n=136, 85.5%) were aware of AUDRA and those participants scored higher on performance expectancy, effort expectancy, and social influence. Overall, the model explains 35.4% of the variance in the behavioral intention to use AUDRA and 11.1% of the variance in use. Social influence (P=.31), especially for women (P=.23), and effort expectancy (P=.25) were key antecedents of behavioral intention. Facilitating conditions were not significant overall but were moderated by age (P=.23), suggesting that it matters for older participants. Performance expectancy did not predict behavioral intention, which is unlike many other technologies but confirms other findings associated with mobile health (mHealth). Open-ended questions suggest that privacy concerns may significantly influence the use of AUDRA. CONCLUSIONS This study suggests that unlike many other technologies, the adoption of AUDRA is not mainly determined by utilitarian factors such as performance expectancy. Rather, effort expectancy and social influence play a key role in determining the intention to use AUDRA.
- Open Access
- 14 July 2021
- Published by Springer Nature in Research Square
Impact of Artificial Intelligence, With and Without Information, on Pathologists’ Decisions: An Experiment
Background: Artificial intelligence (AI) is rapidly gaining attention in medicine and in pathology in particular. While much progress has been made in refining the accuracy of algorithms, thereby increasing their potential use, we need to better understand how these algorithms will be used by pathologists, who will remain for the foreseeable future the decision-makers. The objective of this paper is to determine the propensity of pathologists to rely on AI decision aids and to investigate whether providing information on the algorithm impacts this reliance.Methods: To test our hypotheses, we conducted an experiment with within-subjects design using an online survey study. 116 respondent pathologists and pathology students participated in the experiment. Each participant was tasked with assessing the Gleason grade for a series of 12 prostate cancer samples under three conditions: without advice, with advice from an AI decision aid, and with advice from an AI decision aid with information provided on the algorithm, namely the algorithm accuracy rate and the algorithm model. Scores were computed by comparing the respondents’ scores with the “true” score at the individual-question level. A mixed effects logistic regression was used to analyze the difference in scores between the different conditions, controlling for the random effects of participants and images and to assess the interactions with Experience, Gender and beliefs towards AI.Results: Participant responses to the questions with AI decision aids were significantly more accurate than the control condition without aid. However, no significant difference was found when subjects were provided with additional accuracy rate and model information on the AI advice. Moreover, the propensity to rely on AI was found to relate to general beliefs on AI but not with particular assessments of the AI tool offered. Males also performed better in the No-aid condition but not in the AI-aid condition.Conclusions: AI can significantly influence pathologists and the general beliefs in AI could be major predictors of future reliance on AI by pathologists.
- 14 July 2021
- Published by Springer Nature in Research Square
Open AccessImpact of Artificial Intelligence, With and Without Information, on Pathologists’ Decisions: An Experiment
Background: Artificial intelligence (AI) is rapidly gaining attention in medicine and in pathology in particular. While much progress has been made in refining the accuracy of algorithms, thereby increasing their potential use, we need to better understand how these algorithms will be used by pathologists, who will remain for the foreseeable future the decision-makers. The objective of this paper is to determine the propensity of pathologists to rely on AI decision aids and to investigate whether providing information on the algorithm impacts this reliance.Methods: To test our hypotheses, we conducted an experiment with within-subjects design using an online survey study. 116 respondent pathologists and pathology students participated in the experiment. Each participant was tasked with assessing the Gleason grade for a series of 12 prostate cancer samples under three conditions: without advice, with advice from an AI decision aid, and with advice from an AI decision aid with information provided on the algorithm, namely the algorithm accuracy rate and the algorithm model. Scores were computed by comparing the respondents’ scores with the “true” score at the individual-question level. A mixed effects logistic regression was used to analyze the difference in scores between the different conditions, controlling for the random effects of participants and images and to assess the interactions with Experience, Gender and beliefs towards AI.Results: Participant responses to the questions with AI decision aids were significantly more accurate than the control condition without aid. However, no significant difference was found when subjects were provided with additional accuracy rate and model information on the AI advice. Moreover, the propensity to rely on AI was found to relate to general beliefs on AI but not with particular assessments of the AI tool offered. Males also performed better in the No-aid condition but not in the AI-aid condition.Conclusions: AI can significantly influence pathologists and the general beliefs in AI could be major predictors of future reliance on AI by pathologists.
- 3 July 2021
- Published by Springer Nature
- p. 600-607
An Experiment on the Impact of Information on the Trust in Artificial Intelligence
- 3 July 2021
- Published by Springer Nature
- p. 600-607
An Experiment on the Impact of Information on the Trust in Artificial Intelligence
- Open Access
- 6 April 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Evaluation of an Online System for Routine Outcome Monitoring: Cross-sectional Survey Study (Preprint)
BACKGROUND The use of routine outcome monitoring (ROM) in the treatment of mental health has emerged as a method of improving psychotherapy treatment outcomes. Despite this, very few clinicians regularly use ROM in clinical practice. Online ROM has been suggested as a solution to increase adoption. OBJECTIVE The aim of this study is to identify the influence of moving ROM online on client completion rates of self-reported outcome measures and to identify implementation and utilization barriers to online ROM by assessing clinicians’ views on their experience using the online system over previous paper-based methods. METHODS Client completion rates of self-reported outcome measures were compared pre- and postimplementation of an online system of ROM. In addition, a survey questionnaire was administered to 324 mental health service providers regarding their perception of the benefits with an online system of ROM. RESULTS Client completion rates of self-reported measures increased from 15.62% (427/2734) to 53.98% (1267/2347) after they were moved online. Furthermore, 57% (56/98) of service providers found the new system less time-consuming than the previous paper-based ROM, and 64% (63/98) found that it helped monitor clients. However, the perceived value of the system remains in doubt as only 23% (23/98) found it helped them identify clients at risk for treatment failure, and only 18% (18/98) found it strengthened the therapeutic alliance. CONCLUSIONS Although the current study suggests mixed results regarding service providers’ views on their experience using an online system for ROM, it has identified barriers and challenges that are actionable for improvement.
- 6 April 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessEvaluation of an Online System for Routine Outcome Monitoring: Cross-sectional Survey Study (Preprint)
BACKGROUND The use of routine outcome monitoring (ROM) in the treatment of mental health has emerged as a method of improving psychotherapy treatment outcomes. Despite this, very few clinicians regularly use ROM in clinical practice. Online ROM has been suggested as a solution to increase adoption. OBJECTIVE The aim of this study is to identify the influence of moving ROM online on client completion rates of self-reported outcome measures and to identify implementation and utilization barriers to online ROM by assessing clinicians’ views on their experience using the online system over previous paper-based methods. METHODS Client completion rates of self-reported outcome measures were compared pre- and postimplementation of an online system of ROM. In addition, a survey questionnaire was administered to 324 mental health service providers regarding their perception of the benefits with an online system of ROM. RESULTS Client completion rates of self-reported measures increased from 15.62% (427/2734) to 53.98% (1267/2347) after they were moved online. Furthermore, 57% (56/98) of service providers found the new system less time-consuming than the previous paper-based ROM, and 64% (63/98) found that it helped monitor clients. However, the perceived value of the system remains in doubt as only 23% (23/98) found it helped them identify clients at risk for treatment failure, and only 18% (18/98) found it strengthened the therapeutic alliance. CONCLUSIONS Although the current study suggests mixed results regarding service providers’ views on their experience using an online system for ROM, it has identified barriers and challenges that are actionable for improvement.