Christopher A. Abeare

Objective: The evaluation of self-reported symptoms is a standard component of concussion assessment and management. Clinicians typically evaluate a total symptom severity score rather than scores corresponding to specific symptom domains (i.e., cognitive, sleep-arousal, physical, and affective symptoms). This study examined (i) whether elevations in specific symptom domains would be missed when interpreting only the total symptom severity score and (ii) if a single symptom domain elevation was more common than having elevated symptoms across multiple domains. Method: Adolescent student-athletes (N = 1,008) with concussion history (i.e., ≥6 months since last concussion) completed the Post-Concussion Symptom Scale (PCSS). The PCSS total score and cognitive, sleep-arousal, physical, and affective domain scores were calculated. To determine if symptoms were elevated, scores were compared to normative data matched on gender and pre-existing conditions, with scores considered elevated if they were ≥84th percentile. The frequency of total and domain score elevations were calculated and stratified by gender and number of prior concussions (i.e., 1 or ≥2 prior concussions). Results: Overall, 26% of student-athletes had an elevated symptom domain score without being elevated on the total score. The most common symptom presentation was to have a single elevated symptom domain (21%), followed by two (11%), three (8%), or four elevated domains (6%). Conclusions: Interpreting PCSS symptom domains may be beneficial in detecting student-athletes with elevated symptoms following a remote concussion. Roughly a quarter of student-athletes have domain-specific symptom elevations that would be missed by interpreting the total score alone.

To assess noncredible performance on the NIH Toolbox Cognitive Battery (NIHTB-CB), we developed embedded validity indicators (EVIs). Data were collected from 98 adults (54.1% female) as part of a prospective multicenter cross-sectional study at 4 mild traumatic brain injury (mTBI) specialty clinics. Traditional EVIs and novel item-based EVIs were developed for the NIHTB-CB using the Medical Symptom Validity Test (MSVT) as criterion. The signal detection profile of individual EVIs varied greatly. Multivariate models had superior classification accuracy. Failing >= 4 traditional EVIs at the liberal cutoff or >= 3 at the conservative cutoff produced a good combination of sensitivity (.57 to .61) and specificity (.92 to .94) to MSVT. Combining the traditional and item-based EVIs improved sensitivity (.65 to .70) at comparable specificity (.91 to .95). In conclusion, newly developed EVIs within the NIHTB-CB effectively discriminated between patients who passed versus failed the MSVT. Aggregating EVIs within the same category into validity composites improved signal detection over univariate cutoffs. Item-based EVIs improved classification accuracy over that of traditional EVIs. However, the marginal gains hardly justify the burden of extra calculations. The newly introduced EVIs require cross-validation before wide-spread research or clinical application. Public Significance Statement In this study, we developed the first embedded validity indices within the NIH Toolbox Cognitive Battery in a sample of people with a recent history of mild traumatic brain injury. Embedded validity indices provide information about performance validity, or the degree to which test scores reflect test-taker ability on cognitive tests.

Objective: Performance validity assessment is an important component of concussion baseline testing and Immediate Post-Concussion and Cognitive Testing (ImPACT) is the most commonly used test in this setting. A review of invalid performance on ImPACT was published in 2017, focusing largely on the default embedded validity indicator (Default EVI) provided within the test. There has since been a proliferation in research evaluating the classification accuracy of the Default EVI against independently developed, alternative ImPACT-based EVIs, necessitating an updated review. The purpose of this study was to provide an up-to-date review of the prevalence of invalid performance on ImPACT and to examine the relative effectiveness of ImPACT-based EVIs. Method: Literature related to the prevalence of invalid performance on ImPACT and the effectiveness of ImPACT-based EVIs, published between January 2000 and May 2020, was critically reviewed. Results: A total of 23 studies reported prevalence of invalid performance at baseline testing using ImPACT. Six percent of baseline assessments were found to be invalid by the ImPACT’s Default EVI, and between 22.31% and 34.99% were flagged by alternative EVIs. Six studies assessed the effectiveness of ImPACT-based EVIs, with the Default EVI correctly identifying experimental malingerers only 60% of the time. Alternative ImPACT-based EVIs identified between 73% and 100% of experimental malingerers. Conclusions: The ImPACT’s Default EVI is not sufficiently sensitive, and clinicians should consider alternative indicators when assessing invalid performance. Accordingly, the base rate of invalid performance in athletes at baseline testing is likely well above the 6% previously reported.

Despite growing affective-memory research, only 2 potential clinical measures have been published, each with limitations. We describe the development and piloting of an integrated memory measure for neutral and affectively valenced words, the Cognitive-Affective Verbal Learning Test (C-AVLT). The C-AVLT and mood self-report measures were administered to 124 healthy university students in Study 1, with readministration to 40 students after 1 week. In Study 2, the C-AVLT and other neuropsychological measures of memory and emotion were administered to 61 patients referred for polysomnogram evaluation of obstructive sleep apnea (OSA). Study 1 supported the C-AVLT's internal and test-retest reliabilities, as well as concurrent validity, that is, the affective-bias scores but not performance scores correlated with self-reported mood. In Study 2, convergent, criterion (specifically cross-sectional concurrent validity), and incremental validity were supported with regard to both performance and affective-bias scores within the OSA sample. We demonstrated the C-AVLT is a reliable and clinically useful measure of both memory and affective-processing bias in 2 samples. Future clinical and research recommendations for the C-AVLT are discussed, including broadening normative data and criterion validity data in psychiatric and neurological samples. (PsycINFO Database Record

Research suggests that select processing speed measures can also serve as embedded validity indicators (EVIs). The present study examined the diagnostic utility of Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) subtests as EVIs in a mixed clinical sample of 205 patients medically referred for neuropsychological assessment (53.3% female, mean age = 45.1). Classification accuracy was calculated against 3 composite measures of performance validity as criterion variables. A PSI ≤79 produced a good combination of sensitivity (.23-.56) and specificity (.92-.98). A Coding scaled score ≤5 resulted in good specificity (.94-1.00), but low and variable sensitivity (.04-.28). A Symbol Search scaled score ≤6 achieved a good balance between sensitivity (.38-.64) and specificity (.88-.93). A Coding-Symbol Search scaled score difference ≥5 produced adequate specificity (.89-.91) but consistently low sensitivity (.08-.12). A 2-tailed cutoff on the Coding/Symbol Search raw score ratio (≤1.41 or ≥3.57) produced acceptable specificity (.87-.93), but low sensitivity (.15-.24). Failing ≥2 of these EVIs produced variable specificity (.81-.93) and sensitivity (.31-.59). Failing ≥3 of these EVIs stabilized specificity (.89-.94) at a small cost to sensitivity (.23-.53). Results suggest that processing speed based EVIs have the potential to provide a cost-effective and expedient method for evaluating the validity of cognitive data. Given their generally low and variable sensitivity, however, they should not be used in isolation to determine the credibility of a given response set. They also produced unacceptably high rates of false positive errors in patients with moderate-to-severe head injury. Combining evidence from multiple EVIs has the potential to improve overall classification accuracy. (PsycINFO Database Record

Objective: This study compared failure rates on performance validity tests (PVTs) across liberal and conservative cutoffs in a sample of undergraduate students participating in academic research.Method: Participants (n = 120) were administered four free-standing PVTs (Test of Memory Malingering, Word Memory Test, Rey 15-Item Test, Hiscock Forced-Choice Procedure) and three embedded PVTs (Digit Span, letter and category fluency). Participants also reported their perceived level of effort during testing.Results: At liberal cutoffs, 36.7% of the sample failed ≥1 PVTs, 6.7% failed ≥2, and .8% failed 3. At conservative cutoffs, 18.3% of the sample failed ≥1 PVTs, 2.5% failed ≥2, and .8% failed 3. Participants were 3 to 5 times more likely to fail embedded (15.8–30.8%) compared to free-standing PVTs (3.3–10.0%). There was no significant difference in failure rates between native and non-native English speaking participants at either liberal or conservative cutoffs. Additionally, there was no relation between self-reported effort and PVT failure rates.Conclusions: Although PVT failure rates varied as a function of PVTs and cutoffs, between a third and a fifth of the sample failed ≥1 PVTs, consistent with high initial estimates of invalid performance in this population. Embedded PVTs had notably higher failure rates than free-standing PVTs. Assuming optimal effort in research using students as participants without a formal assessment of performance validity introduces a potentially significant confound in the study design.