MKMegan O. Kelly

Megan O. Kelly

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Megan O. Kelly has a total of 14 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    The effect of source reliability and information credibility on judgments of information quality in intelligence analysis

    by Megan O. Kelly, David V. Budescu, Mandeep Dhami, David R. Mandel

    The quality of information that informs decisions in expert domains such as law enforcement and national security often requires assessment based on meta-informational attributes such as source reliability and information credibility. Across 2 experiments with intelligence analysts (n = 74) and nonexperts (n = 175), participants rated the accuracy, informativeness, trustworthiness, and usefulness of information varying in source reliability and information credibility. The latter 2 attributes were communicated using ratings from the Admiralty Code, an information-evaluation system widely used in the defence and security domain since the 1940s. Ratings of accuracy, informativeness, and likelihood of use were elicited as repeated measures to examine intraindividual reliability. Across experiments, intraindividual reliability was best when levels of source reliability and information credibility were moderately consistent compared to when they were maximally inconsistent (i.e., one low and one high) or maximally consistent (both high or low). As well, trustworthiness ratings depended more on source reliability than on information credibility. Finally, the likelihood of using information was consistently predicted by accuracy ratings and not by judged informativeness or trustworthiness. The current findings offer insights into the ability of experts and novices to reliably use information-evaluation systems for structuring human judgments about intelligence.


  • Open Access

    The Effect of Source Reliability and Information Credibility on Judgments of Information Quality in Intelligence Analysis

    by Megan O. Kelly, David Budescu, Mandeep K Dhami, David R. Mandel

    The quality of information that informs decisions in expert domains such as law enforcement and national security often requires assessment based on meta-informational attributes such as source reliability and information credibility. Across two experiments with intelligence analysts (n = 74) and non-experts (n = 175), participants rated the accuracy, informativeness, trustworthiness and usefulness of information varying in source reliability and information credibility. The latter two attributes were communicated using ratings from the Admiralty Code, an information evaluation system widely used in the defence and security domain since the 1940s. Ratings of accuracy, informativeness, and likelihood of use were elicited as repeated measures to examine intra-individual reliability. Across experiments, intra-individual reliability was best when levels of source reliability and information credibility were moderately consistent compared to when they were maximally inconsistent (i.e., one low and one high) or maximally consistent (both high or low). As well, trustworthiness ratings depended more on source reliability than on information credibility. Finally, the likelihood of using information was consistently predicted by accuracy ratings and not by judged informativeness or trustworthiness. The current findings offer insights into the ability of experts and novices to reliably use information evaluation systems for structuring human judgments about intelligence.

  • Does expecting external memory support cost recognition memory?

    by Megan O. Kelly, Batul Karimjee, April E. Pereira, Xinyi Lu, Evan F. Risko

    We often use tools and aids to help us achieve our cognitive goals – that is, we often offload to external supports. One such variety of offloading i


  • Open Access

    The effect of calibration training on the calibration of intelligence analysts' judgments

    by Megan O. Kelly, David R. Mandel

    Experts are expected to make well‐calibrated judgments within their field, yet a voluminous literature demonstrates miscalibration in human judgment. Calibration training aimed at improving subsequent calibration performance offers a potential solution. We tested the effect of commercial calibration training on a group of 70 intelligence analysts by comparing the miscalibration and bias of their judgments before and after a commercial training course meant to improve calibration across interval estimation and binary choice tasks. Training significantly improved calibration and bias overall, but this effect was contingent on the task. For interval estimation, analysts were overconfident before training and became better calibrated after training. For the binary choice task, however, analysts were initially underconfident and bias increased in this same direction post‐training. Improvement on the two tasks was also uncorrelated. Taken together, results indicate that the training shifted analyst bias toward less confidence rather than having improved metacognitive monitoring ability.


  • Open Access

    When Half Is at Least 50%: Effect of “Framing” and Probability Level on Frequency Estimates

    by David R. Mandel, Megan Kelly

    Expert judgment often involves estimating magnitudes, such as the frequency of deaths due to a pandemic. Three experiments (Ns = 902, 431, and 755, respectively) were conducted to examine the effect of outcome framing (e.g., half of a threatened group expected to survive vs. die), probability level (low vs. high), and probability format (verbal, numeric, or combined) on the estimated frequency of survivals/deaths. Each experiment found an interactive effect of frame and probability level, which supported the hypothesis that forecasted outcomes received by participants were implicitly quantified as lower bounds (i.e., “at least half”). Responding in a manner consistent with a lower‐bound “at least” interpretation was unrelated to incoherence (Experiments 1 and 2) and positively related to numeracy (Experiments 1 and 3), verbal reasoning (Experiment 3), and actively open‐minded thinking (Experiments 2 and 3). The correlational results indicate that implicit lower bounding is an aspect of linguistic inference and not a cognitive error. Implications for research on framing effects are discussed.


  • Productions Need Not Match Study Items to Confer a Production Advantage, But It Helps

    by Megan O. Kelly, Xinyi Lu, Tyler M. Ensor, Colin M. MacLeod, Evan F. Risko

    The production effect is the finding that, relative to silent reading, producing information at study (e.g., reading aloud) leads to a benefit in memory. In most studies of this effect, individuals are presented with a set of unique items, and they produce a subset of these items (e.g., they are presented with the to-be-remembered target item TABLE and produce table) such that the production is both unique and representative of the target. Across two preregistered experiments, we examined the influence of a production that is unique but that does not match the target (e.g., producing fence to the target TABLE, producing car to the target TREE, and so on). This kind of production also yielded a significant effect—the mismatching production effect—although it was smaller than the standard production effect (i.e., when productions are both unique and representative of their targets) and was detectable only when targets with standard productions were included in the same study phase (i.e., when the type of production was manipulated within participant). We suggest that target-production matching is an important precursor to the production effect and that the kind of production that brings about a benefit depends on the other productions that are present.


  • Open Access

    Effect of Calibration Training on the Calibration of Intelligence Analysts’ Judgments

    by Megan O. Kelly, David R. Mandel

    Experts are expected to make well-calibrated judgments within their field, yet a voluminous literature demonstrates miscalibration in human judgment. Calibration training aimed at improving subsequent calibration performance offers a potential solution. We tested the effect of commercial calibration training on a group of 70 intelligence analysts by comparing the miscalibration and bias of their judgments before and after a commercial training course meant to improve calibration across interval estimation and binary choice tasks. Training significantly improved calibration and bias overall, but this effect was contingent on the task. For interval estimation, analysts were overconfident before training and became better calibrated after training. For the binary choice task, however, analysts were initially underconfident and bias increased in this same direction post-training. Improvement on the two tasks was also uncorrelated. Taken together, results indicate that the training shifted analyst bias toward less confidence rather than improve metacognitive monitoring ability.

  • The prod eff: Partially producing items moderates the production effect

    by Megan O. Kelly, Tyler M. Ensor, Colin M. MacLeod, Evan F. Risko

    Current accounts of the production effect suggest that production leads to the encoding of additional production-associated features and/or better feature


  • Open Access

    Thinking in the digital age: Everyday cognition and the dawn of a new age of metacognition research

    by Evan F. Risko, Megan O. Kelly

    The special issue “Rethinking Cognition in a Digital Age” features a diverse set of contributions examining interactions between cognition and technology. In this commentary, we draw connections between this work and similar efforts in the everyday cognition tradition that emphasizes the importance of examining cognition in context. In addition, using the present contributions as inspiration, we suggest that one of the key directions for future research will be metacognitive. That is, with technology increasingly performing cognitive functions for us, understanding how we monitor and control internal and external cognitive resources to achieve our goals represents an important and exciting new frontier in cognitive psychology.