Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 9691-9709
Abstract
Michael Y. Hu, Jackson Petty, Chuan Shi, William Merrill, Tal Linzen. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- Bigger is not always better: The importance of human-scale language modeling for psycholinguisticsJournal of Memory and Language, 2025
- Scaling Backwards: Minimal Synthetic Pre-Training?2024
- Mapping the Early Language Environment Using All-Day Recordings and Automated AnalysisAmerican Journal of Speech-Language Pathology, 2017
- The Nature of Statistical Learning Theory2000
- Eigenvalues of covariance matrices: Application to neural-network learningPhysical Review Letters, 1991
- On uniformity within NC1Journal of Computer and System Sciences, 1990
- Classes of languages and linear-bounded automataInformation and Control, 1964
- On certain formal properties of grammarsInformation and Control, 1959
- The Algebraic Theory of Context-Free Languages1959
- Three models for the description of languageIEEE Transactions on Information Theory, 1956