Megan O. Kelly
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Megan O. Kelly has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- David R. Mandel
Intelligence, Influence, and Collaboration Section Defence Research and Development Canada Toronto Ontario Canada
6 shared publications Evan F. Risko
6 shared publications- Colin M. MacLeod
Department of Psychology, University of Waterloo, Waterloo, Canada
2 shared publications - Evan F. Risko
Department of Psychology, University of Waterloo, Waterloo, Canada
2 shared publications - Tyler M. Ensor
California State University, Bakersfield, USA
2 shared publications - Xinyi Lu
Department of Psychology, University of Waterloo, Waterloo, Canada
2 shared publications
Latest Publications
- Open Access
- 12 September 2025
- Published by Cambridge University Press (CUP) in Judgment and Decision Making
- Vol. 20, e36
The effect of source reliability and information credibility on judgments of information quality in intelligence analysis
The quality of information that informs decisions in expert domains such as law enforcement and national security often requires assessment based on meta-informational attributes such as source reliability and information credibility. Across 2 experiments with intelligence analysts (n = 74) and nonexperts (n = 175), participants rated the accuracy, informativeness, trustworthiness, and usefulness of information varying in source reliability and information credibility. The latter 2 attributes were communicated using ratings from the Admiralty Code, an information-evaluation system widely used in the defence and security domain since the 1940s. Ratings of accuracy, informativeness, and likelihood of use were elicited as repeated measures to examine intraindividual reliability. Across experiments, intraindividual reliability was best when levels of source reliability and information credibility were moderately consistent compared to when they were maximally inconsistent (i.e., one low and one high) or maximally consistent (both high or low). As well, trustworthiness ratings depended more on source reliability than on information credibility. Finally, the likelihood of using information was consistently predicted by accuracy ratings and not by judged informativeness or trustworthiness. The current findings offer insights into the ability of experts and novices to reliably use information-evaluation systems for structuring human judgments about intelligence.
- 12 September 2025
- Published by Cambridge University Press (CUP) in Judgment and Decision Making
- Vol. 20, e36
Open AccessThe effect of source reliability and information credibility on judgments of information quality in intelligence analysis
The quality of information that informs decisions in expert domains such as law enforcement and national security often requires assessment based on meta-informational attributes such as source reliability and information credibility. Across 2 experiments with intelligence analysts (n = 74) and nonexperts (n = 175), participants rated the accuracy, informativeness, trustworthiness, and usefulness of information varying in source reliability and information credibility. The latter 2 attributes were communicated using ratings from the Admiralty Code, an information-evaluation system widely used in the defence and security domain since the 1940s. Ratings of accuracy, informativeness, and likelihood of use were elicited as repeated measures to examine intraindividual reliability. Across experiments, intraindividual reliability was best when levels of source reliability and information credibility were moderately consistent compared to when they were maximally inconsistent (i.e., one low and one high) or maximally consistent (both high or low). As well, trustworthiness ratings depended more on source reliability than on information credibility. Finally, the likelihood of using information was consistently predicted by accuracy ratings and not by judged informativeness or trustworthiness. The current findings offer insights into the ability of experts and novices to reliably use information-evaluation systems for structuring human judgments about intelligence.
- Open Access
- 19 June 2025
- Published by Center for Open Science in PsyArXiv
The Effect of Source Reliability and Information Credibility on Judgments of Information Quality in Intelligence Analysis
The quality of information that informs decisions in expert domains such as law enforcement and national security often requires assessment based on meta-informational attributes such as source reliability and information credibility. Across two experiments with intelligence analysts (n = 74) and non-experts (n = 175), participants rated the accuracy, informativeness, trustworthiness and usefulness of information varying in source reliability and information credibility. The latter two attributes were communicated using ratings from the Admiralty Code, an information evaluation system widely used in the defence and security domain since the 1940s. Ratings of accuracy, informativeness, and likelihood of use were elicited as repeated measures to examine intra-individual reliability. Across experiments, intra-individual reliability was best when levels of source reliability and information credibility were moderately consistent compared to when they were maximally inconsistent (i.e., one low and one high) or maximally consistent (both high or low). As well, trustworthiness ratings depended more on source reliability than on information credibility. Finally, the likelihood of using information was consistently predicted by accuracy ratings and not by judged informativeness or trustworthiness. The current findings offer insights into the ability of experts and novices to reliably use information evaluation systems for structuring human judgments about intelligence.
- 19 June 2025
- Published by Center for Open Science in PsyArXiv
Open AccessThe Effect of Source Reliability and Information Credibility on Judgments of Information Quality in Intelligence Analysis
The quality of information that informs decisions in expert domains such as law enforcement and national security often requires assessment based on meta-informational attributes such as source reliability and information credibility. Across two experiments with intelligence analysts (n = 74) and non-experts (n = 175), participants rated the accuracy, informativeness, trustworthiness and usefulness of information varying in source reliability and information credibility. The latter two attributes were communicated using ratings from the Admiralty Code, an information evaluation system widely used in the defence and security domain since the 1940s. Ratings of accuracy, informativeness, and likelihood of use were elicited as repeated measures to examine intra-individual reliability. Across experiments, intra-individual reliability was best when levels of source reliability and information credibility were moderately consistent compared to when they were maximally inconsistent (i.e., one low and one high) or maximally consistent (both high or low). As well, trustworthiness ratings depended more on source reliability than on information credibility. Finally, the likelihood of using information was consistently predicted by accuracy ratings and not by judged informativeness or trustworthiness. The current findings offer insights into the ability of experts and novices to reliably use information evaluation systems for structuring human judgments about intelligence.
- 31 January 2025
- Published by Springer Nature in Memory & Cognition
- Vol. 53 (6) , 1865-1887
Does expecting external memory support cost recognition memory?
We often use tools and aids to help us achieve our cognitive goals – that is, we often offload to external supports. One such variety of offloading i
- 31 January 2025
- Published by Springer Nature in Memory & Cognition
- Vol. 53 (6) , 1865-1887
Does expecting external memory support cost recognition memory?
We often use tools and aids to help us achieve our cognitive goals – that is, we often offload to external supports. One such variety of offloading i
- Open Access
- 7 September 2024
- Published by Wiley in Applied Cognitive Psychology
- Vol. 38 (5)
The effect of calibration training on the calibration of intelligence analysts' judgments
Experts are expected to make well‐calibrated judgments within their field, yet a voluminous literature demonstrates miscalibration in human judgment. Calibration training aimed at improving subsequent calibration performance offers a potential solution. We tested the effect of commercial calibration training on a group of 70 intelligence analysts by comparing the miscalibration and bias of their judgments before and after a commercial training course meant to improve calibration across interval estimation and binary choice tasks. Training significantly improved calibration and bias overall, but this effect was contingent on the task. For interval estimation, analysts were overconfident before training and became better calibrated after training. For the binary choice task, however, analysts were initially underconfident and bias increased in this same direction post‐training. Improvement on the two tasks was also uncorrelated. Taken together, results indicate that the training shifted analyst bias toward less confidence rather than having improved metacognitive monitoring ability.
- 7 September 2024
- Published by Wiley in Applied Cognitive Psychology
- Vol. 38 (5)
Open AccessThe effect of calibration training on the calibration of intelligence analysts' judgments
Experts are expected to make well‐calibrated judgments within their field, yet a voluminous literature demonstrates miscalibration in human judgment. Calibration training aimed at improving subsequent calibration performance offers a potential solution. We tested the effect of commercial calibration training on a group of 70 intelligence analysts by comparing the miscalibration and bias of their judgments before and after a commercial training course meant to improve calibration across interval estimation and binary choice tasks. Training significantly improved calibration and bias overall, but this effect was contingent on the task. For interval estimation, analysts were overconfident before training and became better calibrated after training. For the binary choice task, however, analysts were initially underconfident and bias increased in this same direction post‐training. Improvement on the two tasks was also uncorrelated. Taken together, results indicate that the training shifted analyst bias toward less confidence rather than having improved metacognitive monitoring ability.
- Open Access
- 9 July 2024
- Published by Wiley in Journal of Behavioral Decision Making
- Vol. 37 (3)
When Half Is at Least 50%: Effect of “Framing” and Probability Level on Frequency Estimates
Expert judgment often involves estimating magnitudes, such as the frequency of deaths due to a pandemic. Three experiments (Ns = 902, 431, and 755, respectively) were conducted to examine the effect of outcome framing (e.g., half of a threatened group expected to survive vs. die), probability level (low vs. high), and probability format (verbal, numeric, or combined) on the estimated frequency of survivals/deaths. Each experiment found an interactive effect of frame and probability level, which supported the hypothesis that forecasted outcomes received by participants were implicitly quantified as lower bounds (i.e., “at least half”). Responding in a manner consistent with a lower‐bound “at least” interpretation was unrelated to incoherence (Experiments 1 and 2) and positively related to numeracy (Experiments 1 and 3), verbal reasoning (Experiment 3), and actively open‐minded thinking (Experiments 2 and 3). The correlational results indicate that implicit lower bounding is an aspect of linguistic inference and not a cognitive error. Implications for research on framing effects are discussed.
- 9 July 2024
- Published by Wiley in Journal of Behavioral Decision Making
- Vol. 37 (3)
Open AccessWhen Half Is at Least 50%: Effect of “Framing” and Probability Level on Frequency Estimates
Expert judgment often involves estimating magnitudes, such as the frequency of deaths due to a pandemic. Three experiments (Ns = 902, 431, and 755, respectively) were conducted to examine the effect of outcome framing (e.g., half of a threatened group expected to survive vs. die), probability level (low vs. high), and probability format (verbal, numeric, or combined) on the estimated frequency of survivals/deaths. Each experiment found an interactive effect of frame and probability level, which supported the hypothesis that forecasted outcomes received by participants were implicitly quantified as lower bounds (i.e., “at least half”). Responding in a manner consistent with a lower‐bound “at least” interpretation was unrelated to incoherence (Experiments 1 and 2) and positively related to numeracy (Experiments 1 and 3), verbal reasoning (Experiment 3), and actively open‐minded thinking (Experiments 2 and 3). The correlational results indicate that implicit lower bounding is an aspect of linguistic inference and not a cognitive error. Implications for research on framing effects are discussed.
- 1 January 2024
- Published by Hogrefe Publishing Group in Experimental Psychology
- Vol. 71 (1) , 2-13
Productions Need Not Match Study Items to Confer a Production Advantage, But It Helps
The production effect is the finding that, relative to silent reading, producing information at study (e.g., reading aloud) leads to a benefit in memory. In most studies of this effect, individuals are presented with a set of unique items, and they produce a subset of these items (e.g., they are presented with the to-be-remembered target item TABLE and produce table) such that the production is both unique and representative of the target. Across two preregistered experiments, we examined the influence of a production that is unique but that does not match the target (e.g., producing fence to the target TABLE, producing car to the target TREE, and so on). This kind of production also yielded a significant effect—the mismatching production effect—although it was smaller than the standard production effect (i.e., when productions are both unique and representative of their targets) and was detectable only when targets with standard productions were included in the same study phase (i.e., when the type of production was manipulated within participant). We suggest that target-production matching is an important precursor to the production effect and that the kind of production that brings about a benefit depends on the other productions that are present.
- 1 January 2024
- Published by Hogrefe Publishing Group in Experimental Psychology
- Vol. 71 (1) , 2-13
Productions Need Not Match Study Items to Confer a Production Advantage, But It Helps
The production effect is the finding that, relative to silent reading, producing information at study (e.g., reading aloud) leads to a benefit in memory. In most studies of this effect, individuals are presented with a set of unique items, and they produce a subset of these items (e.g., they are presented with the to-be-remembered target item TABLE and produce table) such that the production is both unique and representative of the target. Across two preregistered experiments, we examined the influence of a production that is unique but that does not match the target (e.g., producing fence to the target TABLE, producing car to the target TREE, and so on). This kind of production also yielded a significant effect—the mismatching production effect—although it was smaller than the standard production effect (i.e., when productions are both unique and representative of their targets) and was detectable only when targets with standard productions were included in the same study phase (i.e., when the type of production was manipulated within participant). We suggest that target-production matching is an important precursor to the production effect and that the kind of production that brings about a benefit depends on the other productions that are present.
- Open Access
- 22 December 2023
- Published by Center for Open Science in PsyArXiv
Effect of Calibration Training on the Calibration of Intelligence Analysts’ Judgments
Experts are expected to make well-calibrated judgments within their field, yet a voluminous literature demonstrates miscalibration in human judgment. Calibration training aimed at improving subsequent calibration performance offers a potential solution. We tested the effect of commercial calibration training on a group of 70 intelligence analysts by comparing the miscalibration and bias of their judgments before and after a commercial training course meant to improve calibration across interval estimation and binary choice tasks. Training significantly improved calibration and bias overall, but this effect was contingent on the task. For interval estimation, analysts were overconfident before training and became better calibrated after training. For the binary choice task, however, analysts were initially underconfident and bias increased in this same direction post-training. Improvement on the two tasks was also uncorrelated. Taken together, results indicate that the training shifted analyst bias toward less confidence rather than improve metacognitive monitoring ability.
- 22 December 2023
- Published by Center for Open Science in PsyArXiv
Open AccessEffect of Calibration Training on the Calibration of Intelligence Analysts’ Judgments
Experts are expected to make well-calibrated judgments within their field, yet a voluminous literature demonstrates miscalibration in human judgment. Calibration training aimed at improving subsequent calibration performance offers a potential solution. We tested the effect of commercial calibration training on a group of 70 intelligence analysts by comparing the miscalibration and bias of their judgments before and after a commercial training course meant to improve calibration across interval estimation and binary choice tasks. Training significantly improved calibration and bias overall, but this effect was contingent on the task. For interval estimation, analysts were overconfident before training and became better calibrated after training. For the binary choice task, however, analysts were initially underconfident and bias increased in this same direction post-training. Improvement on the two tasks was also uncorrelated. Taken together, results indicate that the training shifted analyst bias toward less confidence rather than improve metacognitive monitoring ability.
- 24 August 2023
- Published by Springer Nature in Psychonomic Bulletin & Review
- Vol. 31 (1) , 373-379
The prod eff: Partially producing items moderates the production effect
Current accounts of the production effect suggest that production leads to the encoding of additional production-associated features and/or better feature
- 24 August 2023
- Published by Springer Nature in Psychonomic Bulletin & Review
- Vol. 31 (1) , 373-379
The prod eff: Partially producing items moderates the production effect
Current accounts of the production effect suggest that production leads to the encoding of additional production-associated features and/or better feature
- Open Access
- 25 June 2023
- Published by Wiley in Applied Cognitive Psychology
- Vol. 37 (4) , 785-788
Thinking in the digital age: Everyday cognition and the dawn of a new age of metacognition research
The special issue “Rethinking Cognition in a Digital Age” features a diverse set of contributions examining interactions between cognition and technology. In this commentary, we draw connections between this work and similar efforts in the everyday cognition tradition that emphasizes the importance of examining cognition in context. In addition, using the present contributions as inspiration, we suggest that one of the key directions for future research will be metacognitive. That is, with technology increasingly performing cognitive functions for us, understanding how we monitor and control internal and external cognitive resources to achieve our goals represents an important and exciting new frontier in cognitive psychology.
- 25 June 2023
- Published by Wiley in Applied Cognitive Psychology
- Vol. 37 (4) , 785-788
Open AccessThinking in the digital age: Everyday cognition and the dawn of a new age of metacognition research
The special issue “Rethinking Cognition in a Digital Age” features a diverse set of contributions examining interactions between cognition and technology. In this commentary, we draw connections between this work and similar efforts in the everyday cognition tradition that emphasizes the importance of examining cognition in context. In addition, using the present contributions as inspiration, we suggest that one of the key directions for future research will be metacognitive. That is, with technology increasingly performing cognitive functions for us, understanding how we monitor and control internal and external cognitive resources to achieve our goals represents an important and exciting new frontier in cognitive psychology.