The “analysis of competing hypotheses” in intelligence analysis
Open Access
- 21 March 2019
- journal article
- research article
- Published by Wiley in Applied Cognitive Psychology
- Vol. 33 (6) , 1080-1090
Abstract
Summary: The intelligence community uses “structured analytic techniques” to help analysts think critically and avoid cognitive bias. However, little evidence exists of how techniques are applied and whether they are effective. We examined the use of the analysis of competing hypotheses (ACH)—a technique designed to reduce “confirmation bias.” Fifty intelligence analysts were randomly assigned to use ACH or not when completing a hypothesis testing task that had probabilistic ground truth. Data on analysts' judgement processes and conclusions were collected using written protocols that were then coded for statistical analyses. We found that ACH‐trained analysts did not follow all of the steps of ACH. There was mixed evidence for ACH's ability to reduce confirmation bias, and we observed that ACH may increase judgement inconsistency and error. It may be prudent for the intelligence community to consider the conditions under which ACH would prove useful and to explore alternatives.Keywords
Funding Information
- Government of the United Kingdom (NA)
This publication has 34 references indexed in Scilit:
- On avoiding framing effects in experienced decision makersQuarterly Journal of Experimental Psychology, 2013
- Pictures speak louder than numbers: on communicating medical risks to immigrants with limited non‐native language proficiencyHealth Expectations, 2011
- Training and Educating U.S. Intelligence AnalystsInternational Journal of Intelligence and CounterIntelligence, 2008
- Finding Useful Questions: On Bayesian Diagnosticity, Probability, Impact, and Information Gain.Psychological Review, 2005
- Separating the wheat from the chaff: does discriminating between diagnostic and nondiagnostic information eliminate the dilution effect?Journal of Behavioral Decision Making, 2004
- The Effect of Experience on the Use of Irrelevant Evidence in Auditor JudgmentThe Accounting Review, 1999
- Confirmation Bias: A Ubiquitous Phenomenon in Many GuisesReview of General Psychology, 1998
- How to improve Bayesian reasoning without instruction: Frequency formats.Psychological Review, 1995
- Confirmation and Matching Biases in Hypothesis TestingThe Quarterly Journal of Experimental Psychology Section A, 1988
- On the psychology of prediction.Psychological Review, 1973