Christopher Draheim

Aviation selection tests are high-stakes assessments designed to identify candidates capable of succeeding in demanding flight environments. Most branches of the US military incorporate both content-based and process-based assessments to evaluate prior knowledge and reasoning ability, respectively. A challenge with high-stakes process tests is that their validity requires participant naivety, which is increasingly difficult to maintain in the modern internet era. As such, these high-stakes tests must be continuously evaluated to ensure the most valid, reliable, and cost-effective selection procedures are employed. To address this, we examined practice effects in the psychomotor airplane tracking tasks of the Navy’s Aviation Selection Test Battery (ASTB). We had 146 Naval Flight Students and 119 enlisted Sailors perform the ASTB’s psychomotor battery six times across two days. Results revealed large practice effects, shifting in rank ordering of individuals across attempts, and that psychomotor performance had not stabilized even by the sixth attempt. Prior action video gaming and flight simulator gaming experience correlated with psychomotor performance, with some evidence that improvements were related, albeit weakly, to either gaming experience or initial performance. Finally, correlations between psychomotor performance and eventual naval flight training scores were stable across the six attempts, but simulations indicated predictive validity can range widely if participants have differing levels of practice from one another. Overall, these findings indicate that the psychomotor component of the ASTB is a valuable inclusion to the Naval Flight School selection process but also could benefit from further refinement.

Objective: We explored transfer of learning from brief practice with different input devices in the Navy’s Performance Based Measures Battery (PBM), a psychomotor subset of the Aviation Selection Test Battery (ASTB). Background: The PBM is a set of computerized tests used as a part of the ASTB to select aviators in the U.S. military. Official practice is not available, leading candidates to practice with unofficial re-creations and with or without access to the stick and throttle used on the PBM. Method: Our between-subjects study with 152 cadets from the U.S. Military Academy evaluated the impact of mouse/keyboard or stick/throttle practice on the psychomotor portions of the PBM compared to a control group that was only presented with an informational video. Results: The results showed that practice with either input device resulted in improved performance relative to control on the PBM’s two-dimensional airplane tracking task (ATT). For the simpler vertical tracking task (VTT), the mouse/keyboard group showed significantly worse performance than either stick/throttle practice or control groups, indicating a transfer cost from practicing with an alternative input device. Conclusion: The results suggest that becoming familiar with the unique dynamics of the ATT may be more important than practicing with the appropriate input device. Conversely, device-specific motor learning appears to be a more impactful determinant of performance for the simpler VTT. This indicates that transfer effects from alternative input devices depend in part on properties of the task. Application: This research can inform practice policies for psychomotor test selection.

There is an increasing consensus among researchers that traditional attention tasks do not validly index the attentional mechanisms that they are often used to assess. We recently tested and validated several existing, modified, and new tasks and found that accuracy-based and adaptive tasks were more reliable and valid measures of attention control than traditional ones, which typically rely on speeded responding and/or contrast comparisons in the form of difference scores (Draheim et al. Journal of Experimental Psychology: General, 150(2), 242-275, 2021). With these improved measures, we found that attention control fully mediated the working memory capacity-fluid intelligence relationship, a novel finding that we argued has significant theoretical implications. The present study was both a follow-up and extension to this "toolbox approach" to measuring attention control. Here, we tested updated versions of several attention control tasks in a new dataset (N = 301) and found, with one exception, that these tasks remain strong indicators of attention control. The present study also replicated two important findings: (1) that attention control accounted for nearly all the variance in the relationship between working memory capacity and fluid intelligence, and (2) that the strong association found between attention control and other cognitive measures is not because the attention control tasks place strong demands on processing speed. These findings show that attention control can be measured as a reliable and valid individual differences construct, and that attention control shares substantial variance with other executive functions.

There is an increasing consensus among researchers that traditional attention tasks do not validly index the attentional mechanisms that they are often used to assess. We recently tested and validated several existing, modified, and new tasks and found that accuracy-based and adaptive tasks were more reliable and valid measures of attention control than traditional ones, which typically rely on speeded responding and/or contrast comparisons in the form of difference scores (Draheim et al., 2021). With these improved measures, we found that attention control fully mediated the working memory capacity-fluid intelligence relationship, a novel finding that we argued has significant theoretical implications. The present study was both a follow-up and extension to this “toolbox approach” to measuring attention control. Here, we tested updated versions of several attention control tasks in a new dataset (N = 301) and found, with one exception, that these tasks remain strong indicators of attention control. The present study also replicated two important findings: 1) that attention control accounted for nearly all the variance in the relationship between working memory capacity and fluid intelligence, and 2) that the strong association found between attention control and other cognitive measures is not because the attention control tasks place strong demands on processing speed. These findings show that attention control can be measured as a reliable and valid individual differences construct and that attention control shares substantial variance with other executive functions.

Working memory capacity is an important psychological construct, and many real-world phenomena are strongly associated with individual differences in worki

Working memory capacity is an important psychological construct and many real-world phenomena are strongly associated with individual differences in working memory functioning. Although working memory and attention are intertwined, several studies have recently shown that individual differences in the general ability to control attention is more strongly predictive of human behavior than working memory capacity. In this review, we argue that researchers would therefore generally be better suited to studying the role of attention control rather than memory-based abilities in explaining real-world behavior and performance in humans. The review begins with a discussion of relevant literature on the nature and measurement of both working memory capacity and attention control, including recent developments in the study of individual differences of attention control. We then selectively review existing literature on the role of both working memory and attention in various applied settings and explain, in each case, why a switch in emphasis to attention control is warranted. Topics covered include psychological testing, cognitive training, education, sports, police decision making, human factors, and disorders within clinical psychology. The review concludes with general recommendations and best practices for researchers interested in conducting studies of individual differences in attention control.

Cognitive tasks that produce reliable and robust effects at the group level often fail to yield reliable and valid individual differences. An ongoing debate among attention researchers is whether conflict resolution mechanisms are task-specific or domain-general, and the lack of correlation between most attention measures seems to favor the view that attention control is not a unitary concept. We have argued that the use of difference scores, particularly in reaction time (RT), is the primary cause of null and conflicting results at the individual differences level, and that methodological issues with existing tasks preclude making strong theoretical conclusions. The present article is an empirical test of this view in which we used a toolbox approach to develop and validate new tasks hypothesized to reflect attention processes. Here, we administered existing, modified, and new attention tasks to over 400 participants (final N = 396). Compared with the traditional Stroop and flanker tasks, performance on the accuracy-based measures was more reliable, had stronger intercorrelations, formed a more coherent latent factor, and had stronger associations to measures of working memory capacity and fluid intelligence. Further, attention control fully accounted for the relationship between working memory capacity and fluid intelligence. These results show that accuracy-based measures can be better suited to individual differences investigations than traditional RT tasks, particularly when the goal is to maximize prediction. We conclude that attention control is a unitary concept. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Why do some individuals learn more quickly than others, or perform better in complex cognitive tasks? In this article, we describe how differential and experimental research methods can be used to study intelligence in humans and non-human animals. More than one hundred years ago, Spearman (1904) discovered a general factor underpinning performance across cognitive domains in humans. Shortly thereafter, Thorndike (1935) discovered positive correlations between cognitive performance measures in the albino rat. Today, research continues to shed light on the underpinnings of the positive manifold observed among ability measures. In this review, we focus on the relationship between cognitive performance and attention control: the domain-general ability to maintain focus on task-relevant information while preventing attentional capture by task-irrelevant thoughts and events. Recent work from our lab has revealed that individual differences in attention control can largely explain the positive associations between broad cognitive abilities such as working memory capacity and fluid intelligence. In research on mice, attention control has been closely linked to a general ability factor reflecting route learning and problem solving. Taken together, both lines of research suggest that individual differences in attention control underpin performance in a variety of complex cognitive tasks, helping to explain why measures of cognitive ability correlate positively. Efforts to find confirmatory and disconfirmatory evidence across species stands to improve not only our understanding of attention control, but cognition in general.

Attentional control as an ability to regulate information processing during goal-directed behavior is critical to many theories of human cognition and thought to predict a large range of everyday behaviors. However, in recent years, failures to reliably assess individual differences in attentional control have sparked a debate concerning whether attentional control, as currently conceptualized and assessed, can be regarded as a valid psychometric construct. In this consensus paper, we summarize the current debate from theoretical, methodological, and analytical perspectives. First, we propose a consensus-based definition of attentional control and the cognitive mechanisms that potentially contribute to individual differences in attentional control. Next, guided by the findings of an in-depth literature survey, we discuss the psychometric considerations that are critical when assessing attentional control. We then provide suggestions for recent methodological and analytical approaches that can alleviate the most common concerns. We conclude that, to truly advance our understanding of the construct of attentional control, we must develop a theory-driven and empirically supported consensus on how we define, operationalize, and assess attentional control. This consensus paper presents a first step to achieve this goal; a shift toward transparent reporting, sharing of materials and data, and cross-laboratory efforts will further accelerate progress.