The need for quantitative methods in syntax and semantics research
Top Cited Papers
Open Access
- 1 January 2013
- journal article
- research article
- Published by Taylor & Francis in Language and Cognitive Processes
- Vol. 28 (1-2) , 88-124
Abstract
The prevalent method in syntax and semantics research involves obtaining a judgement of the acceptability of a sentence/meaning pair, typically by just the author of the paper, sometimes with feedback from colleagues. This methodology does not allow proper testing of scientific hypotheses because of (a) the small number of experimental participants (typically one); (b) the small number of experimental stimuli (typically one); (c) cognitive biases on the part of the researcher and participants; and (d) the effect of the preceding context (e.g., other constructions the researcher may have been recently considering). In the current paper we respond to some arguments that have been given in support of continuing to use the traditional nonquantitative method in syntax/semantics research. One recent defence of the traditional method comes from Phillips (2009), who argues that no harm has come from the nonquantitative approach in syntax research thus far. Phillips argues that there are no cases in the literature where an incorrect intuitive judgement has become the basis for a widely accepted generalisation or an important theoretical claim. He therefore concludes that there is no reason to adopt more rigorous data collection standards. We challenge Philips' conclusion by presenting three cases from the literature where a faulty intuition has led to incorrect generalisations and mistaken theorising, plausibly due to cognitive biases on the part of the researchers. Furthermore, we present additional arguments for rigorous data collection standards. For example, allowing lax data collection standards has the undesirable effect that the results and claims will often be ignored by researchers with stronger methodological standards. Finally, we observe that behavioural experiments are easier to conduct in English than ever before, with the advent of Amazon.com's Mechanical Turk, a marketplace interface that can be used for collecting behavioural data over the internet.Keywords
This publication has 41 references indexed in Scilit:
- Acoustic correlates of information structureLanguage and Cognitive Processes, 2010
- Naive v. expert intuitions: An empirical study of acceptability judgmentsThe Linguistic Review, 2010
- Empirical evidence and theoretical reasoning in generative grammarTheoretical Linguistics, 2007
- Carrots – perfect as vegetables, but please not as a main dishTheoretical Linguistics, 2007
- Data in generative grammar: The stick and the carrotTheoretical Linguistics, 2007
- A few lessons from typologyLinguistic Typology, 2007
- Computational principles of working memory in sentence comprehensionTrends in Cognitive Sciences, 2006
- Magnitude estimation and what it can do for your syntax: some wh-constraints in GermanLingua, 2005
- Gradience in linguistic dataLingua, 2005
- Reading relative clauses in EnglishCognitive Linguistics, 2005