Kristy A. Martire

Determining which experts to trust is essential for both routine and high-stakes decisions, yet evaluating expertise can be difficult. In this Review, we examine the cognitive processes that underpin genuine expertise and explore the disconnect between psychological insights into expertise and the practical methods used to evaluate it. In settings where expertise must be evaluated by laypeople, such as adversarial legal trials, evaluators face substantial challenges, including knowledge disparities that hinder analysis, communication barriers that impact the clear explanation of expert methods, and procedural constraints that limit the scrutiny of expert evidence. These challenges complicate the assessment of expert claims and contribute to wrongful convictions and unjust outcomes. We suggest that a distinction between ‘show-it’ and ‘know-it’ expert performances that differ in their visibility, measurability, and immediacy can be used as a heuristic for identifying when evaluations of expertise require greater care and should incorporate a variety of diagnostic factors including foundational and applied validity. Finally, we highlight key knowledge gaps and propose promising directions for future research to improve evaluations of expertise in a range of contexts.

This protocol aims to address two metaresearch questions: what do we know about the science underlying impactful legal decisions and how can we assess this evidence efficiently and accurately, such that it is usable for courts? To do so, this protocol describes our plan for a systematic review of the literature underlying whether there are reliable indicators of the manner of death in stab wound cases. Evidence of nature was recently found admissible at the High Court of Australia in Lang v The Queen. However, despite the High Court uncritically accepting this evidence, there has never been a systematic review that critically appraises the relevant literature. Beyond the review itself, we will fully document our process and all of the time-saving tools we rely on to provide knowledge about whether systematic reviews of this nature can be efficiently deployed in legal proceedings.

This document describes the method and data used to inform Figure 2 in the paper “Psychological Insights For Judging Experts”.

People vary substantially in their ability to visually compare or ‘match’ patterns – some are much more accurate than others. This is a varied and generalisable ability in the general population – for example, novices who excel in fingerprint-matching also excel in firearms-matching. Forensic feature-comparison examiners also excel in this task, but no research has investigated the generalisability of their skill. The extent to which this superiority will generalise and predict their ability in related tasks is unclear. In this study, we investigated the generalisability of perceptual expertise amongst forensic examiners by comparing their performance to novices and other examiners within and outside their area of expertise. We recruited 85 experts from three forensic disciplines (face, fingerprint, and firearms) and asked them to complete four different visual comparison tasks: faces, fingerprints, firearms, and novel objects. Examiners displayed domain-specific expertise: they outperformed novices and other examiners within their domain of visual comparison expertise. Yet some of their skill also generalised: examiners also outperformed novices outside their area of expertise. These results provide key insight into the domain-specific and domain-general contributions of forensic examiners’ perceptual expertise. Forensic expertise lends some generalisable skill to other visual comparison tasks – but domain expertise still leads to the best performance.