Testing diachronic measures of productivity using the Zipf-Mandelbrot law
- 13 October 2025
- book chapter
- Published by John Benjamins Publishing Company
- p. 6-16
Abstract
The productivity of patterns (be it morphological or syntactic) has long been studied in diachrony. However, the study of the relationship between productivity and token frequency has been hindered by the non-trivial dependency of productivity on the sample size on which it is computed. Several solutions have been offered in the literature but a systematic evaluation of them all has not been performed to this date. In this paper, we rely on the Zipf-Mandelbrot distribution to generate synthetic data and perform a statistical evaluation of these methods. Our result is that many of these methods are successful at providing a measure independent of the sample size, and that they are closely correlated.Keywords
This publication has 17 references indexed in Scilit:
- The hapax / type ratioCorpus Studies of Language Through Time, 2022
- The predictive capabilities of mathematical models for the type-token relationship in English language corporaComputer Speech & Language, 2021
- Productivity, richness, and diversity of light verb constructions in the history of American EnglishJournal of Historical Linguistics, 2020
- Productivity in Argument Selection2012
- Zipf's Law Leads to Heaps' Law: Analyzing Their Relation in Finite-Size SystemsPLOS ONE, 2010
- Construction Learning as a Function of Frequency, Frequency Distribution, and FunctionThe Modern Language Journal, 2009
- 41. Corpus linguistics in morphology: Morphological productivity2009
- The Emergence of Productive Non-Medical -itis: Corpus Evidence and Qualitative Analysis2005
- Using a corpus of children's writing to test a solution to the sample size problem affecting type-token ratios2004
- Word Frequency Distributions2001