Michel Rathbone

Background: People who sustain a concussion and live in remote areas can experience challenges in accessing specialized assessments. In these cases, virtual approaches to assessment are of value. There is limited information on important psychometric properties of physical assessment measures used to evaluate people postconcussion virtually. Objective: The aims of this method-comparison psychometric study were to determine (1) inter- and intrarater reliability of a battery of concussion physical tests administered virtually in people with brain injury and (2) sensitivity and specificity of the virtual battery when compared to the in-person assessment. Methods: A total of 60 people living with acquired brain injuries attended an in-person and virtual assessment at the Ottawa Hospital Rehabilitation Centre. The order of the assessments, in-person and virtual, was randomized. The following physical measures were administered in-person and virtually: finger-to-nose test, vestibular ocular motor screening (VOMS), static balance testing (double leg, single leg, and tandem), saccades, cervical spine range of motion, and evaluation of effort. The virtual assessment was recorded, and a second clinician viewed and independently documented findings from the recordings twice at 1-month intervals. Results: The mean age of the participants was 45.65 (SD 16.50) years. The sensitivity metrics ranged from moderate (60%, 95% CI 30-86) to excellent (100%, 95% CI 71-100) for saccades and cervical spine right lateral flexion, respectively. Specificity ranged from 75%, 95% CI 35-95 to 100%, 95% CI 91-100 for left single leg stance eyes closed and left finger-to-nose testing, respectively. The interrater reliability ranged from poor for cervical spine extension (Cohen κ=0.20, 95% CI −0.07 to 0.47) to excellent for VOMS change in symptoms (Cohen κ=0.93, 95% CI 0.83-1). The intrarater reliability ranged from poor for cervical spine extension (Cohen κ=0.31, 95% CI 0.04-0.58) to excellent for the finger-to-nose testing on the right (Cohen κ=0.90, 95% CI 0.71-1). The wide CIs highlight variability in precision and suggest that further research with larger samples is needed before clinical use can be fully standardized. Conclusions: This study provides information on the psychometric properties associated with virtual administration of concussion measures. The VOMS change in symptoms measure appears to have the most promising properties when administered virtually when in-person visits are not possible. This is particularly relevant for patients in rural areas, for those facing access barriers, and in contexts where timely follow-up is challenging. However, caution should be maintained when administering certain concussion measures virtually. The wide CIs for some measures caution against over-reliance on single test findings, and clinicians should consider both the strengths and limitations of virtual delivery. Clinicians are encouraged to make informed decisions about which measures can be effectively used remotely, and which may still require in-person administration to maintain accuracy.

BACKGROUND People who sustain a concussion and live in remote areas can experience challenges to accessing specialized assessments. In these cases, virtual approaches to assessment are of value. There is limited information on important psychometric properties of physical assessment measures used to evaluate people post-concussion virtually. OBJECTIVE The objectives of this method-comparison study were to determine i) inter/intra-rater reliability of a battery of concussion physical tests administered virtually in people with brain injury and ii) sensitivity and specificity of the virtual battery when compared to the in-person assessment. METHODS Sixty people living with acquired brain injuries attended an in-person and virtual assessment, at the Ottawa Hospital Rehabilitation Centre. The order of the assessments, in-person and virtual, was randomized. The following physical measures were administered in person and virtually: finger-to-nose test, Vestibular/Ocular Motor Screening (VOMS), static balance testing (double leg, single leg, tandem), saccades, cervical spine range of motion, and evaluation of effort. The virtual assessment was recorded and a second clinician viewed and independently documented findings from the recordings twice at one-month intervals. RESULTS The sensitivity metrics ranged from moderate (60%) to excellent (100%) for saccades and cervical spine lateral flexion measures, respectively. Specificity ranged from 75% to 100% for left single leg stance eyes closed and left finger-to-nose testing, respectively. The inter-rater reliability ranged from poor for cervical spine extension (Cohen’s Kappa = 0.20) to excellent for VOMS change in symptoms (Cohen’s Kappa = 0.93). The intra-rater reliability ranged from poor for cervical spine extension (Cohen’s Kappa = 0.31) to excellent for the finger-to-nose testing on the right (Cohen’s Kappa = 0.90). CONCLUSIONS This study provides information on the psychometric properties associated with virtual administration of concussion measures. The VOMS change in symptoms measure appears to have most promising properties when administered virtually. Caution should be maintained when administering certain concussion measures virtually. INTERNATIONAL REGISTERED REPORT RR2-10.2196/57663

Background: Remote approaches to workplace concussion assessment have demonstrated value to end users. The feasibility of administering physical concussion assessment measures in a remote context has been minimally explored, and there is limited information on important psychometric properties of physical assessment measures used in remote contexts. Objective: The objectives of this feasibility study were to determine recruitment capability for a future larger-scale study aimed at determining sensitivity and reliability of the remote assessment, time required to complete study assessments, and acceptability of remote assessment to people with brain injuries and clinicians; document preliminary results of the sensitivity of the remote assessment when compared to the in-person assessment; and estimate the preliminary interrater and intrarater reliability of the remote assessments to inform procedures of a future larger-scale study that is adequately powered to reliably estimate these parameters of interest. Methods: People living with acquired brain injury attended 2 assessments (1 in-person and 1 remote) in a randomized order. The measures administered in these assessments included the finger-to-nose test; balance testing; and the Vestibular/Ocular Motor Screening (VOMS) tool, including documentation of change in symptoms and distance for near point convergence, saccades, cervical spine range of motion, and evaluation of effort. Both assessments occurred at the Ottawa Hospital Rehabilitation Center. After the assessments, a clinician different from the person who completed the original assessments then viewed and documented findings independently on the recordings of the remote assessment. The same second clinician viewed the recording again approximately 1 month following the initial observation. Results: The rate of recruitment was 61% (20/33) of people approached, with a total of 20 patient-participants included in the feasibility study. A total of 3 clinicians participated as assessors. The length of time required to complete the in-person and remote assessment procedures averaged 9 and 13 minutes, respectively. The majority of clinicians and patient-participants agreed or strongly agreed that they were confident in the findings on both in-person and remote assessments. Feedback obtained revolved around technology (eg, screen size), lighting, and fatigue of participants in the second assessment. Preliminary estimates of sensitivity of the remote assessment ranged from poor (finger-to-nose testing: 0.0) to excellent (near point convergence: 1.0). Preliminary estimates of reliability of the remote assessment ranged from poor (balance testing, saccades, and range of motion: κ=0.38‐0.49) to excellent (VOMS change in symptoms: κ=1.0). Conclusions: The results of this feasibility study indicate that our study procedures are feasible and acceptable to participants. Certain measures show promising psychometric properties (reliability and sensitivity); however, wide CIs due to the small sample size limit the ability to draw definitive conclusions. A planned follow-up study will expand on this work and include a sufficiently large sample to estimate these important properties with acceptable precision.

Background: Evaluating the clinical status of concussions using virtual platforms has become increasingly common. While virtual approaches to care are useful, there is limited information regarding the barriers and facilitators associated with a virtual concussion assessment. Objective: This study aims to identify the barriers and facilitators associated with engaging in virtual concussion assessments from the perspective of people living with workplace concussions; identify the barriers and facilitators to completing virtual concussion assessments from the perspectives of clinicians; and identify the clinical measures related to 4 clinical domains that would be most appropriate in virtual practice: general neurological examination and vestibular, oculomotor, and cervical spine assessment. We also evaluated effort. Methods: Separate online focus groups were conducted with expert concussion clinicians and people living with workplace concussions. A moderator led the focus groups using a semistructured interview guide that targeted a discussion of participants’ experiences with virtual assessments. The discussions were recorded, transcribed, and analyzed by 2 reviewers using content analysis. Barriers and facilitators associated with completing the physical concussion examination were categorized based on the domain of the concussion examination and more general barriers and facilitators. Clinician-selected measures believed to work best in a virtual practice were described using frequency counts. Results: A total of 4 focus groups with 15 people living with workplace concussions and 3 focus groups with 14 clinicians were completed using Microsoft Teams. Barriers were identified, such as triggering of symptoms associated with completing an assessment over video (mentioned 13/162 (8%) and 9/201 (4%) of the time for patient and clinician participants, respectively); challenges with location and setup (mentioned 16/162 (10%) of the time for patient participants); communication (mentioned 34/162 (21%) and 9/201 (4%) of the time for patient and clinician participants, respectively); and safety concerns (mentioned 11/162 (7%) of the time for patient and 15/201 (7%) for clinician participants). Facilitators were identified, such as having access to support (mentioned 42/154 (27%) and 21/151 (14%) of the time for patient and clinician participants, respectively); implementing symptom management strategies throughout the assessment (mentioned 11/154 (7%) of the time for patient participants); and having access to resources (mentioned 25/151 (17%) of the time for clinician participants). From the perspective of the clinician participants included in this study, the clinical measures recommended most for a virtual practice were finger to nose testing; balance testing; the Vestibular/Ocular Motor Screening tool; saccades; and cervical spine range of motion within their respective domains (ie, neurological examination, vestibular, oculomotor, and cervical spine assessment). Conclusions: Virtual assessments appear to be useful for both people living with workplace concussions and clinicians. While barriers were identified, such as challenges associated with exposure to screens, virtual assessments have benefits such as improved access to care. The clinician-selected measures that were considered best in a virtual practice will be investigated in an upcoming evaluative study. International Registered Report Identifier (IRRID): RR2-10.2196/40446

Background: There is little guidance available, and no uniform assessment battery is used in either in-person or remote evaluations of people who are experiencing persistent physical symptoms post concussion. Selecting the most appropriate measures for both in-person and remote physical assessments is challenging because of the lack of expert consensus and guidance. Objective: This study used expert consensus processes to identify clinical measures currently used to assess 5 physical domains affected by concussion (neurological examination, cervical spine, vestibular, oculomotor, or effort) and determine the feasibility of applying the identified measures virtually. Methods: The Delphi approach was used. In the first round, experienced clinicians were surveyed regarding using measures in concussion assessment. In the second round, clinicians reviewed information regarding the psychometric properties of all measures identified in the first round by at least 15% (9/58) of participants. In the second round, experts rank-ordered the measures from most relevant to least relevant based on their clinical experience and documented psychometric properties. A working group of 4 expert clinicians then determined the feasibility of virtually administering the final set of measures. Results: In total, 59 clinicians completed survey round 1 listing all measures they used to assess the physical domains affected by a concussion. The frequency counts of the 146 different measures identified were determined. Further, 33 clinicians completed the second-round survey and rank-ordered 22 measures that met the 15% cutoff criterion retained from round 1. Measures ranked first were coordination, range of motion, vestibular ocular motor screening, and smooth pursuits. These measures were feasible to administer virtually by the working group members; however, modifications for remote administration were recommended, such as adjusting the measurement method. Conclusions: Clinicians ranked assessment of coordination (finger-to-nose test and rapid alternating movement test), cervical spine range of motion, vestibular ocular motor screening, and smooth pursuits as the most relevant measures under their respective domains. Based on expert opinion, these clinical measures are considered feasible to administer for concussion physical examinations in the remote context, with modifications; however, the psychometric properties have yet to be explored. International Registered Report Identifier (IRRID): RR2-10.2196/40446

Background: Workplace mild traumatic brain injuries are frequently associated with persistent symptoms, leading to a reduction in productivity at work or even disability. People who sustain workplace injuries frequently need rehabilitation and support, and the challenges of delivering these services was heightened during the COVID-19 pandemic as injured workers had to be cared for remotely. Currently, clinicians are conducting both in-person and virtual (remote) concussion assessments; however, the measures that are being used to complete these assessments have undocumented psychometric properties. Objective: This study will document the psychometric properties of the clinical measures that are being used remotely and their ability to produce similar results to in-person assessments. Specifically, through this method-comparison study, we aim to (1) evaluate the sensitivity of the measures included in a virtual assessment toolkit when compared to an in-person assessment and (2) determine the interrater and intrarater reliabilities of the measures included in a virtual assessment toolkit. Methods: Patient participants (people living with acquired brain injuries) will attend two assessments (in person and virtual) at the Ottawa Hospital. The two assessments will be identical, consisting of the measures included in our previously developed virtual concussion assessment toolkit, which includes finger-to-nose testing, the Vestibular/Ocular Motor Screening tool, balance testing, cervical spine range of motion, saccades testing, and evaluation of effort. All virtual assessments will occur using the Microsoft Teams platform and will be audio/video-recorded. The clinician assessor and patient participant will complete a feedback form following completion of the assessments. A different clinician will also document the findings on observed videos of the virtual assessment shortly after completion of both in-person and virtual assessments and approximately 1 month later. Interrater reliability will be assessed by comparing the second clinician’s observation with the first clinician’s initial virtual assessment. Intrarater reliability will be evaluated by comparing the second clinician’s observation with their own assessment approximately 1 month later. Sensitivity will be documented by comparing the findings (identification of abnormality) of the in-person assessment completed by the initial clinician assessor with those of the second clinician assessor on the observation of the recording of the virtual assessment. Results: As of May 2024, we have recruited 7 clinician assessors and completed study assessments with 39 patient participants. The study recruitment is expected to be completed by September 2024. Conclusions: Currently, it is unknown if completing concussion assessments virtually produces similar results to the in-person assessment. This work will serve as a first step to determining the similarity of the virtual assessment to the matching in-person assessment and will provide information on the reliability of the virtual assessment. International Registered Report Identifier (IRRID): DERR1-10.2196/57663

BACKGROUND Virtual approaches to workplace concussion assessment have demonstrated value to end users. The feasibility of administering physical concussion assessment measures in a virtual context has been minimally explored and there is limited information on important psychometric properties of physical assessment measures used in virtual contexts. OBJECTIVE The objectives of this feasibility study were to: determine recruitment capability for a future larger-scale study aimed at determining sensitivity and reliability of the virtual assessment, time required to complete study assessments, acceptability of virtual assessment to people with brain injuries and clinicians, and to document preliminary results of sensitivity of the virtual assessment when compared to the in-person assessment and to estimate the preliminary inter-rater and intra-rater reliability of the virtual assessments to inform procedures of a future larger-scale study that is adequately powered to reliably estimate these parameters of interest. METHODS People living with acquired brain injury attended two assessments (one in-person and one virtual) in a randomized order. The measures administered in these assessments included: the finger-to-nose test, balance testing, the Vestibular/Ocular Motor Screening (VOMS) tool including documentation of change in symptoms and distance for near point convergence (NPC), saccades, cervical spine range of motion, and evaluation of effort. Both assessments occurred at the Ottawa Hospital Rehabilitation Centre. After the assessments, a clinician different from the person who completed the original assessments then viewed and documented findings independently on the recordings of the virtual assessment. The same second clinician viewed the recording again approximately one month following initial observation. RESULTS Rate of recruitment was 61% of people approached, with a total of 20 patient-participants included in the feasibility study. Three clinicians participated as assessors. Length of time required to complete the in-person and virtual assessment procedures averaged nine and 13 minutes, respectively. The majority of clinician and patient-participants agreed or strongly agreed that they were confident in the findings on both in-person and virtual assessments. Feedback obtained revolved around technology (e.g. screen size), lighting, and fatigue of participants in the second assessment. Preliminary estimates of sensitivity of the virtual assessment ranged from poor (finger-to-nose testing: 0.0) to excellent (NPC: 1.0). Preliminary estimates of reliability of the virtual assessment ranged from poor (balance testing, saccades, and range of motion: Kappa=0.38-0.49) to excellent (VOMS change in symptoms: Kappa=1.0). CONCLUSIONS The results of this feasibility study indicate that our study procedures are feasible and acceptable to participants. Certain measures show promising psychometric properties (reliability and sensitivity); however, wide confidence intervals due to the small sample size limit the ability to draw definitive conclusions. A planned follow-up study will expand on this work and include a sufficiently large sample to estimate these important properties with acceptable precision. INTERNATIONAL REGISTERED REPORT RR2-10.2196/preprints.57663

BACKGROUND Workplace mild traumatic brain injuries are frequently associated with persistent symptoms, leading to a reduction in productivity at work or even disability. People who sustain workplace injuries frequently need rehabilitation and support, and the challenges of delivering these services was heightened during the COVID-19 pandemic as injured workers had to be cared for remotely. Currently, clinicians are conducting both in-person and virtual (remote) concussion assessments; however, the measures that are being used to complete these assessments have undocumented psychometric properties. OBJECTIVE This study will document the psychometric properties of the clinical measures that are being used remotely and their ability to produce similar results to in-person assessments. Specifically, through this method-comparison study, we aim to (1) evaluate the sensitivity of the measures included in a virtual assessment toolkit when compared to an in-person assessment and (2) determine the interrater and intrarater reliabilities of the measures included in a virtual assessment toolkit. METHODS Patient participants (people living with acquired brain injuries) will attend two assessments (in person and virtual) at the Ottawa Hospital. The two assessments will be identical, consisting of the measures included in our previously developed virtual concussion assessment toolkit, which includes finger-to-nose testing, the Vestibular/Ocular Motor Screening tool, balance testing, cervical spine range of motion, saccades testing, and evaluation of effort. All virtual assessments will occur using the Microsoft Teams platform and will be audio/video-recorded. The clinician assessor and patient participant will complete a feedback form following completion of the assessments. A different clinician will also document the findings on observed videos of the virtual assessment shortly after completion of both in-person and virtual assessments and approximately 1 month later. Interrater reliability will be assessed by comparing the second clinician’s observation with the first clinician’s initial virtual assessment. Intrarater reliability will be evaluated by comparing the second clinician’s observation with their own assessment approximately 1 month later. Sensitivity will be documented by comparing the findings (identification of abnormality) of the in-person assessment completed by the initial clinician assessor with those of the second clinician assessor on the observation of the recording of the virtual assessment. RESULTS As of May 2024, we have recruited 7 clinician assessors and completed study assessments with 39 patient participants. The study recruitment is expected to be completed by September 2024. CONCLUSIONS Currently, it is unknown if completing concussion assessments virtually produces similar results to the in-person assessment. This work will serve as a first step to determining the similarity of the virtual assessment to the matching in-person assessment and will provide information on the reliability of the virtual assessment. INTERNATIONAL REGISTERED REPORT DERR1-10.2196/57663

BACKGROUND Evaluating the clinical status of concussions using virtual platforms has become increasingly common. While virtual approaches to care are useful, there is limited information regarding the barriers and facilitators associated with a virtual concussion assessment. OBJECTIVE This study aims to identify the barriers and facilitators associated with engaging in virtual concussion assessments from the perspective of people living with workplace concussions; identify the barriers and facilitators to completing virtual concussion assessments from the perspectives of clinicians; and identify the clinical measures related to 4 clinical domains that would be most appropriate in virtual practice: general neurological examination and vestibular, oculomotor, and cervical spine assessment. We also evaluated effort. METHODS Separate online focus groups were conducted with expert concussion clinicians and people living with workplace concussions. A moderator led the focus groups using a semistructured interview guide that targeted a discussion of participants’ experiences with virtual assessments. The discussions were recorded, transcribed, and analyzed by 2 reviewers using content analysis. Barriers and facilitators associated with completing the physical concussion examination were categorized based on the domain of the concussion examination and more general barriers and facilitators. Clinician-selected measures believed to work best in a virtual practice were described using frequency counts. RESULTS A total of 4 focus groups with 15 people living with workplace concussions and 3 focus groups with 14 clinicians were completed using Microsoft Teams. Barriers were identified, such as triggering of symptoms associated with completing an assessment over video (mentioned 13/162 (8%) and 9/201 (4%) of the time for patient and clinician participants, respectively); challenges with location and setup (mentioned 16/162 (10%) of the time for patient participants); communication (mentioned 34/162 (21%) and 9/201 (4%) of the time for patient and clinician participants, respectively); and safety concerns (mentioned 11/162 (7%) of the time for patient and 15/201 (7%) for clinician participants). Facilitators were identified, such as having access to support (mentioned 42/154 (27%) and 21/151 (14%) of the time for patient and clinician participants, respectively); implementing symptom management strategies throughout the assessment (mentioned 11/154 (7%) of the time for patient participants); and having access to resources (mentioned 25/151 (17%) of the time for clinician participants). From the perspective of the clinician participants included in this study, the clinical measures recommended most for a virtual practice were finger to nose testing; balance testing; the Vestibular/Ocular Motor Screening tool; saccades; and cervical spine range of motion within their respective domains (ie, neurological examination, vestibular, oculomotor, and cervical spine assessment). CONCLUSIONS Virtual assessments appear to be useful for both people living with workplace concussions and clinicians. While barriers were identified, such as challenges associated with exposure to screens, virtual assessments have benefits such as improved access to care. The clinician-selected measures that were considered best in a virtual practice will be investigated in an upcoming evaluative study. INTERNATIONAL REGISTERED REPORT RR2-10.2196/40446

BACKGROUND There is little guidance available, and no uniform assessment battery is used in either in-person or remote evaluations of people who are experiencing persistent physical symptoms post concussion. Selecting the most appropriate measures for both in-person and remote physical assessments is challenging because of the lack of expert consensus and guidance. OBJECTIVE This study used expert consensus processes to identify clinical measures currently used to assess 5 physical domains affected by concussion (neurological examination, cervical spine, vestibular, oculomotor, or effort) and determine the feasibility of applying the identified measures virtually. METHODS The Delphi approach was used. In the first round, experienced clinicians were surveyed regarding using measures in concussion assessment. In the second round, clinicians reviewed information regarding the psychometric properties of all measures identified in the first round by at least 15% (9/58) of participants. In the second round, experts rank-ordered the measures from most relevant to least relevant based on their clinical experience and documented psychometric properties. A working group of 4 expert clinicians then determined the feasibility of virtually administering the final set of measures. RESULTS In total, 59 clinicians completed survey round 1 listing all measures they used to assess the physical domains affected by a concussion. The frequency counts of the 146 different measures identified were determined. Further, 33 clinicians completed the second-round survey and rank-ordered 22 measures that met the 15% cutoff criterion retained from round 1. Measures ranked first were coordination, range of motion, vestibular ocular motor screening, and smooth pursuits. These measures were feasible to administer virtually by the working group members; however, modifications for remote administration were recommended, such as adjusting the measurement method. CONCLUSIONS Clinicians ranked assessment of coordination (finger-to-nose test and rapid alternating movement test), cervical spine range of motion, vestibular ocular motor screening, and smooth pursuits as the most relevant measures under their respective domains. Based on expert opinion, these clinical measures are considered feasible to administer for concussion physical examinations in the remote context, with modifications; however, the psychometric properties have yet to be explored. CLINICALTRIAL INTERNATIONAL REGISTERED REPORT RR2-10.2196/40446