CPColleen E. Patton

Colleen E. Patton

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Colleen E. Patton has a total of 18 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • When Labels Lie: Dissociating Source–Scale Bias From Experienced Performance in Decision Support

    by Caden M. Miller, Ebernoe Guzman-Bonilla, Adolfo R. Escobedo, Colleen E. Patton

    Decision support systems (DSS) are often evaluated before users have much direct experience with them. As a result, the way these systems are labeled may shape selection, reliance, and perceived reliability from the outset. The present work examined whether displayed source and scale labels bias DSS use in a timed object-detection task. Participants searched cluttered images for a bat after choosing between advisors differentiated by source, either human-crowd or machine-generated, and scale, either medium or large. Two experiments were conducted: a default-label (DL) experiment, in which displayed labels matched the underlying recommendation source, and a flipped-label (FL) experiment, in which displayed source labels were decoupled from the underlying source. Overall, source and scale labels affected different stages of DSS use. Participants selected larger-scale labels above chance, while machine-labeled assistance was not selected more often than human-crowd-labeled assistance. Compliance was high and closely matched advisor accuracy, with slightly higher compliance for large-labeled than medium-labeled assistance. Response times mainly reflected repeated exposure. Perceived reliability ratings showed a significant label condition × displayed source interaction, and error-type analyses suggested greater vulnerability to false-positive recommendations under a human-crowd label. Together, these results suggest that source and scale labels shape initial selection, trial-level reliance, and post-use reliability judgments in different ways.

  • The Influence of Exposure and Error Type on Estimates of Automation Reliability

    by Colleen E. Patton, Amelia C. Warden, Ebernoe Guzman-Bonilla, Rebecca L. Pharmer

    Objective: We evaluated the sensitivity and calibration of users when estimating the reliability of an automated decision support system as exposure to the system and the saliency of errors varied. Background: Although users are typically sensitive when estimating the reliability of an automated system, it is not known how exposure to a system or error saliency influences the estimate. Viewpoints of the “leaky integrator” and “intuitive statistician” suggest competing outcomes regarding estimates over time. Methods: Participants identified matching images of MegaBlocks with assistance from an automated system. Participants experienced three levels of reliability and three levels of exposure (number of trials). They provided estimates of reliability, trust, and confidence in their estimates for each system. In Experiment 2, the saliency of false alarms and misses was manipulated. In Experiment 3, the saliency of these error types was matched. Results: Reliability estimates were sensitive to true reliability but were lower with more salient errors. Exposure had no impact on the estimates of reliability or trust. Trust and confidence in the estimate of reliability increased with true reliability. Conclusions: Estimates of reliability are generally sensitive to the true reliability of a system, although they can be hindered by memory limitations and/or saliency effects. Estimates of reliability are made very quickly and do not tend to change with exposure. Applications: Training users on a new system may not require extensive exposure, but early interactions with a system should accurately represent the amount and types of errors that the system will make.


  • (Some) Benefits in Operator Decisions to Use AI After Experiencing Optimal Outcomes

    by Colleen E. Patton, Benjamin A. Clegg, Blake C. Davis, Nathaniel Blanchard

    Objective: The current study aimed to explore the impacts of experiencing superior behaviors—accumulating large amounts of evidence and high automation use rates—on subsequent evidence accumulation rates and adaptable (discretionary) automation use decisions in a dynamic decision-making task. Background: Operators prefer to choose when to engage automated support systems but seldom use them appropriately. They also do not typically collect enough evidence to optimize their decision making. This creates suboptimal performance that could benefit from training better behaviors. Method: Participants collected evidence about movement patterns of ships while assisted by a machine learning aid. They were initially required to collect high levels of evidence and use the aid as a form of hands-on training. Then, they chose how much evidence to collect and when to engage the aid. Results: When given the choice, operators collected less evidence and used the automation less often than had been required during training, but improved their performance compared to unaided trials. Conclusion: Providing operators with early experience of superior behavioral strategies can improve their subsequent decisions. This is a promising direction for achieving human–automation team synergy. Applications: Short exposures to optimal behaviors may be a feasible training approach to improve human–automation interactions in contexts where operators want decisional freedom in their interactions.


  • Perceptions of Difficulty in an Automation-Supported Task

    by Jewel G. Padilla, Colleen E. Patton

    Technology users are increasingly given discretion around when to engage automated systems, however, these decisions are often suboptimal, inhibiting performance. Prior work has suggested that this may stem from improper estimates of task difficulty. This study investigated where perceived task difficulty diverged from objective task difficulty (accuracy) and its impact on automation use. In a visual search task, participants identified target objects in cluttered images and could opt to use an automated aid on certain trials. Results indicated that participants were sensitive to differences in difficulty, but they used the automation more often when they rated the task as more difficult. With automation present, performance was equivalent between the easy and difficult conditions, but participants rated the difficult condition as harder. This may suggest a disconnect between perceived difficulty and objective performance measures. These results highlight the importance of considering both subjective and objective difficulty when designing automated systems.


  • Whose Wisdom? Human Biases for Decision Support System Source and Scale

    by Caden M. Miller, Colleen E. Patton, Adolfo R. Escobedo, Ebernoe Guzman-Bonilla

    Users form snap judgments about decision‑support systems (DSS) that guide later reliance. We examined how those judgments depend on DSS source—machine‑learning algorithm (ML) versus wisdom‑of‑the‑crowd (WoC)—and data scale (small, medium, large). Seventy‑eight participants repeatedly chose between paired advisers (e.g., medium‑ML vs. large‑WoC) and then performed a six‑trial visual‑search task with help from their selected advisor. Participants showed no inherent bias for ML or WoC but consistently preferred advisers built on larger datasets. Within six trials, reliability estimates were calibrated to observed accuracy, indicating rapid trust adjustment. Findings suggest users intuitively weighed sample size over provenance when forecasting DSS performance and that well‑sized crowd systems can compete with contemporary ML advisers.


  • Open Access

    The Choice to Use Automation: Improvements from Evidence Accumulation

    by Colleen E. Patton, Benjamin A. Clegg, Blake C. Davis, Turgay Caglar, Caspian Siebert, Nathaniel Blanchard


  • The relationship of trust and dependence

    by Colleen E. Patton, Christopher D. Wickens

    The concepts of automation trust and dependence have often been viewed as closely related and on occasion, have been conflated in the research community. Yet, trust is a cognitive attitude and dependence is a behavioural measure, so it is unsurprising that different factors can affect the two. Here, we review the literature on the correlation between trust and dependence. On average, this correlation across people was quite low, suggesting that people who are more trusting of automation do not necessarily depend upon it more. Separately, we examined experiments that explicitly manipulated the reliability of automation, finding that higher automation reliability increased trust ratings twice as fast as dependence behaviours. This review provides novel quantitative evidence that the two constructs are not strongly correlated. Implications of this work, including potential moderating variables, contexts where trust is still relevant, and considerations of trust measurement, are discussed.


  • Open Access

    Supporting detection of hostile intentions: automated assistance in a dynamic decision-making context

    by Colleen E. Patton, Christopher D. Wickens, C. A. P. Smith, Kayla M. Noble, Benjamin A. Clegg

    In a dynamic decision-making task simulating basic ship movements, participants attempted, through a series of actions, to elicit and identify which one of six other ships was exhibiting either of two hostile behaviors. A high-performing, although imperfect, automated attention aid was introduced. It visually highlighted the ship categorized by an algorithm as the most likely to be hostile. Half of participants also received automation transparency in the form of a statement about why the hostile ship was highlighted. Results indicated that while the aid’s advice was often complied with and hence led to higher accuracy with a shorter response time, detection was still suboptimal. Additionally, transparency had limited impacts on all aspects of performance. Implications for detection of hostile intentions and the challenges of supporting dynamic decision making are discussed.


  • Automation Use: The Roles of Self-Confidence, Trust, and Workload

    by Colleen E. Patton

    Discretionary automation, or the choice to turn automation on or off, is less studied than dependence on always-present (static) automation, even though discretionary automation is becoming more common in real-world environments. While the influences of dependence on static automation are generally known, it is unclear how they translate to discretionary automation use. Using a dual-task warehouse management paradigm, the influence of workload, trust, and self-confidence were investigated. Results indicated that higher trust in the automation than self-confidence in completing the task manually was a strong driver of automation use, whereas workload of a concurrent task had no effect on use. Implications of this work are discussed.


  • I Spy Differences in System Reliability

    by Colleen Patton, Amelia C. Warden, Rebecca Pharmer

    The current experiment examined how the level of exposure to automation of various reliabilities impacts perceptions of reliability and confidence in perceived reliability ratings. Participants viewed images of multicolored Mega blocks presented side-by-side with a recommendation from an automated system indicating whether the images were a match or not. Participants responded whether the automation recommendation was correct or incorrect. The reliability of the automation across blocks was either 50%, 80%, or 90% and participants saw 10, 50, and 100 trials of each reliability. Overall, the level of exposure to the system had no impact on perceptions of automation reliability, or confidence in reliability ratings. Exposure did differentially impact accuracy of automation classification recommendation across the levels of automation reliability. Results also indicated that increases in perceptions of automation reliability increased confidence in the reliability perceptions. These findings have applications for training participants on imperfectly reliable systems.