Zipf-like Statistical Regularities in Molecular Sequence Representations for Chemical Language Models
- 17 August 2026
- journal article
- Published by American Chemical Society (ACS) in The Journal of Physical Chemistry Letters
- Vol. 17 (34) , 9807-9814
Abstract
Large language model (LLM)-based approaches increasingly use molecular strings such as SMILES and SELFIES for molecular generation and property prediction. However, the statistical properties of molecular token distributions have not been systematically characterized. Here, we analyzed rank–frequency distributions of tokens derived by byte-pair encoding (BPE) across large molecular databases. BPE-derived tokens showed approximate Zipf-like rank–frequency scaling across the examined representations and chemical spaces, with fitted slopes moderately steeper than the canonical value of −1, paralleling a statistical pattern widely observed in natural language. Moreover, when BERT models were pretrained using BPE vocabularies with different rank–frequency slopes, the closeness of these slopes to the ideal Zipf value of −1 strongly correlated with performance on molecular property prediction tasks (Pearson r = 0.91, p < 0.001) and remained associated after adjustment for vocabulary size (partial r = 0.86, p < 0.001). Together, these findings show that molecular BPE vocabularies exhibit an approximate Zipf-like rank–frequency regularity and that slope closeness provides an empirical diagnostic for comparing vocabulary sizes within the examined SMILES/SELFIES BPE framework.Keywords
Funding Information
- Ganjiang Innovation Academy, Chinese Academy of Sciences (E455F001)
- Ganjiang Innovation Academy, Chinese Academy of Sciences (E655F001)
- Double Thousand Plan of Jiangxi Province (jxsq2023101060)
- Chinese Academy of Sciences (XDB1220000)
- Key Research and Development Program of Jiangxi Province (20232BBG70003)
- Graduate Program in Physics at UNIFAL-MG (FU2025C0013)
This publication has 39 references indexed in Scilit:
- Learned Conformational Space and Pharmacophore Into Molecular Foundational ModelAdvanced Science, 2026
- Generative AI for the Design of Molecules: Advances and ChallengesJournal of Chemical Information and Modeling, 2025
- RIGR: Resonance-Invariant Graph Representation for Molecular Property PredictionJournal of Chemical Information and Modeling, 2025
- Evaluating Machine Learning Models for Molecular Property Prediction: Performance and Robustness on Out-of-Distribution DataJournal of Chemical Information and Modeling, 2025
- Machine learning-assisted retrosynthesis planning: Current status and future prospectsChinese Journal of Chemical Engineering, 2024
- ZINC-22─A Free Multi-Billion-Scale Database of Tangible Compounds for Ligand DiscoveryJournal of Chemical Information and Modeling, 2023
- Machine intelligence for chemical reaction spaceWIREs Computational Molecular Science, 2022
- Molecular representations for machine learning applications in chemistryInternational Journal of Quantum Chemistry, 2021
- MolGPT: Molecular Generation Using a Transformer-Decoder ModelJournal of Chemical Information and Modeling, 2021
- Molecular fingerprint similarity search in virtual screeningMethods, 2015