Matthew Connelly
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Matthew Connelly has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
Amit Sharma
Microsoft Research India, Bangalore, India
1 shared publication- Arthur Spirling
Professor, Department of Politics, Princeton University, Princeton, NJ, USA
1 shared publication - Duncan J. Watts
Department of Computer and Information Science, University of Pennsylvania, Philadelphia, USA
1 shared publication - Flávio Codeço Coelho
School of Applied Mathematics, Getulio Vargas Foundation, Rio de Janeiro, Brazil
1 shared publication - Huw Price
Trinity College Cambridge UK
1 shared publication - Jack Goetz
Department of Statistics University of Michigan Ann Arbor MI
1 shared publication
Latest Publications
- 3 October 2023
- Published by Springer Nature in Nature
- Vol. 622 (7981) , 31
AI governance must deal with long-term risks as well
- 3 October 2023
- Published by Springer Nature in Nature
- Vol. 622 (7981) , 31
AI governance must deal with long-term risks as well
- 29 July 2021
- Published by Taylor & Francis in Intelligence and National Security
- Vol. 36 (6) , 781-806
New evidence and new methods for analyzing the Iranian revolution as an intelligence failure
The U.S. government failure to predict the Shah’s 1979 fall is a prototypical example of intelligence failure. We complement the close reading of memoirs and State Department documents with computational analysis of the documents in the aggregate, including cables to and from embassies and consulates in Iran. Using several techniques including traffic and sentiment analysis, we find that officials in Iran reported on the protests but did not stress the situation’s severity until too late. D.C. officials were distracted by other events. Text analysis can complement qualitative approaches to more clearly indicate what good intelligence reporting can and cannot achieve.
- 29 July 2021
- Published by Taylor & Francis in Intelligence and National Security
- Vol. 36 (6) , 781-806
New evidence and new methods for analyzing the Iranian revolution as an intelligence failure
The U.S. government failure to predict the Shah’s 1979 fall is a prototypical example of intelligence failure. We complement the close reading of memoirs and State Department documents with computational analysis of the documents in the aggregate, including cables to and from embassies and consulates in Iran. Using several techniques including traffic and sentiment analysis, we find that officials in Iran reported on the protests but did not stress the situation’s severity until too late. D.C. officials were distracted by other events. Text analysis can complement qualitative approaches to more clearly indicate what good intelligence reporting can and cannot achieve.
- 3 June 2019
- Published by Springer Nature in Nature Human Behaviour
- Vol. 3 (9) , 906-912
Predicting history
Can events be accurately described as historic at the time they are happening? Claims of this sort are in effect predictions about the evaluations of future historians; that is, that they will regard the events in question as significant. Here we provide empirical evidence in support of earlier philosophical arguments1 that such claims are likely to be spurious and that, conversely, many events that will one day be viewed as historic attract little attention at the time. We introduce a conceptual and methodological framework for applying machine learning prediction models to large corpora of digitized historical archives. We find that although such models can correctly identify some historically important documents, they tend to overpredict historical significance while also failing to identify many documents that will later be deemed important, where both types of error increase monotonically with the number of documents under consideration. On balance, we conclude that historical significance is extremely difficult to predict, consistent with other recent work on intrinsic limits to predictability in complex social systems2,3. However, the results also indicate the feasibility of developing ‘artificial archivists’ to identify potentially historic documents in very large digital corpora.
- 3 June 2019
- Published by Springer Nature in Nature Human Behaviour
- Vol. 3 (9) , 906-912
Predicting history
Can events be accurately described as historic at the time they are happening? Claims of this sort are in effect predictions about the evaluations of future historians; that is, that they will regard the events in question as significant. Here we provide empirical evidence in support of earlier philosophical arguments1 that such claims are likely to be spurious and that, conversely, many events that will one day be viewed as historic attract little attention at the time. We introduce a conceptual and methodological framework for applying machine learning prediction models to large corpora of digitized historical archives. We find that although such models can correctly identify some historically important documents, they tend to overpredict historical significance while also failing to identify many documents that will later be deemed important, where both types of error increase monotonically with the number of documents under consideration. On balance, we conclude that historical significance is extremely difficult to predict, consistent with other recent work on intrinsic limits to predictability in complex social systems2,3. However, the results also indicate the feasibility of developing ‘artificial archivists’ to identify potentially historic documents in very large digital corpora.
- Open Access
Mining Events with Declassified Diplomatic Documents
Since 1973 the State Department has been using electronic records systems to preserve classified communications. Recently, approximately 1.9 million of these records from 1973-77 have been made available by the U.S. National Archives. While some of these communication streams have periods witnessing an acceleration in the rate of transmission; others do not show any notable patterns in communication intensity. Given the sheer volume of these communications -- far greater than what had been available until now -- scholars need automated statistical techniques to identify the communications that warrant closer study. We develop a statistical framework that can semi-automatically identify from a large corpus of documents a handful that historians would consider more interesting electronic records. Our approach brings together related but distinct statistical concepts from nonparametric signal estimation and statistical hypothesis testing -- which when put together help us identify and analyze various geometrical aspects of the communication streams. Dominant periods of heightened and sustained activities aka bursts, as identified through these methods, correspond well with historical events recognized by standard reference works on the 1970s.
Open AccessMining Events with Declassified Diplomatic Documents
Since 1973 the State Department has been using electronic records systems to preserve classified communications. Recently, approximately 1.9 million of these records from 1973-77 have been made available by the U.S. National Archives. While some of these communication streams have periods witnessing an acceleration in the rate of transmission; others do not show any notable patterns in communication intensity. Given the sheer volume of these communications -- far greater than what had been available until now -- scholars need automated statistical techniques to identify the communications that warrant closer study. We develop a statistical framework that can semi-automatically identify from a large corpus of documents a handful that historians would consider more interesting electronic records. Our approach brings together related but distinct statistical concepts from nonparametric signal estimation and statistical hypothesis testing -- which when put together help us identify and analyze various geometrical aspects of the communication streams. Dominant periods of heightened and sustained activities aka bursts, as identified through these methods, correspond well with historical events recognized by standard reference works on the 1970s.
- Open Access
Using Artificial Intelligence to Identify State Secrets
Whether officials can be trusted to protect national security information has become a matter of great public controversy, reigniting a long-standing debate about the scope and nature of official secrecy. The declassification of millions of electronic records has made it possible to analyze these issues with greater rigor and precision. Using machine-learning methods, we examined nearly a million State Department cables from the 1970s to identify features of records that are more likely to be classified, such as international negotiations, military operations, and high-level communications. Even with incomplete data, algorithms can use such features to identify 90% of classified cables with <11% false positives. But our results also show that there are longstanding problems in the identification of sensitive information. Error analysis reveals many examples of both overclassification and underclassification. This indicates both the need for research on inter-coder reliability among officials as to what constitutes classified material and the opportunity to develop recommender systems to better manage both classification and declassification.
Open AccessUsing Artificial Intelligence to Identify State Secrets
Whether officials can be trusted to protect national security information has become a matter of great public controversy, reigniting a long-standing debate about the scope and nature of official secrecy. The declassification of millions of electronic records has made it possible to analyze these issues with greater rigor and precision. Using machine-learning methods, we examined nearly a million State Department cables from the 1970s to identify features of records that are more likely to be classified, such as international negotiations, military operations, and high-level communications. Even with incomplete data, algorithms can use such features to identify 90% of classified cables with <11% false positives. But our results also show that there are longstanding problems in the identification of sensitive information. Error analysis reveals many examples of both overclassification and underclassification. This indicates both the need for research on inter-coder reliability among officials as to what constitutes classified material and the opportunity to develop recommender systems to better manage both classification and declassification.
- 19 March 2010
- Published by Taylor & Francis in History and Technology
- Vol. 26 (1) , 85-88
Author response: all biopolitics is global
- 19 March 2010
- Published by Taylor & Francis in History and Technology
- Vol. 26 (1) , 85-88
Author response: all biopolitics is global
- 1 March 2006
- Published by Cambridge University Press (CUP) in Journal of Global History
- Vol. 1 (3) , 299-319
To inherit the Earth. Imagining world population, from the yellow peril to the population bomb
This article narrates the development of a set of ideas and provocative imagery about population growth and movement that has shaped the way people think about world politics. It represented humanity in terms of populations that could and should be controlled to prevent degeneration and preserve civilization. During the late nineteenth and twentieth centuries, this discursive tradition supported a series of political projects that aimed to either exclude those deemed able to subsist on less and reproduce more or regulate reproduction worldwide. Conceiving of the world in terms of populations – rather than nation-states – led people to think of new ways in which it might be divided, unsettling diplomatic alignments and alliances. But it also contributed to critiques of state sovereignty, since population problems were said to affect everyone and require a united response. This intellectual history helps illuminate some of the local and parochial reasons why people began to ‘think globally’.
- 1 March 2006
- Published by Cambridge University Press (CUP) in Journal of Global History
- Vol. 1 (3) , 299-319
To inherit the Earth. Imagining world population, from the yellow peril to the population bomb
This article narrates the development of a set of ideas and provocative imagery about population growth and movement that has shaped the way people think about world politics. It represented humanity in terms of populations that could and should be controlled to prevent degeneration and preserve civilization. During the late nineteenth and twentieth centuries, this discursive tradition supported a series of political projects that aimed to either exclude those deemed able to subsist on less and reproduce more or regulate reproduction worldwide. Conceiving of the world in terms of populations – rather than nation-states – led people to think of new ways in which it might be divided, unsettling diplomatic alignments and alliances. But it also contributed to critiques of state sovereignty, since population problems were said to affect everyone and require a united response. This intellectual history helps illuminate some of the local and parochial reasons why people began to ‘think globally’.