Codon language embeddings provide strong signals for use in protein engineering
Top Cited Papers
Open Access
- 23 February 2024
- journal article
- research article
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 6 (2) , 170-179
Abstract
Protein representations from deep language models have yielded state-of-the-art performance across many tasks in computational protein engineering. In recent years, progress has primarily focused on parameter count, with recent models’ capacities surpassing the size of the very datasets they were trained on. Here we propose an alternative direction. We show that large language models trained on codons, instead of amino acid sequences, provide high-quality representations that outperform comparable state-of-the-art models across a variety of tasks. In some tasks, such as species recognition, prediction of protein and transcript abundance or melting point estimation, we show that a language model trained on codons outperforms every other published protein language model, including some that contain over 50 times more parameters. These results indicate that, in addition to commonly studied scale and model complexity, the information content of biological data provides an orthogonal direction to improve the power of machine learning in biology.Keywords
Funding Information
- RCUK | Engineering and Physical Sciences Research Council (EP/T517811/1, EP/W522582/1)
This publication has 53 references indexed in Scilit:
- The UniProtKB guide to the human proteomeDatabase: The Journal of Biological Databases and Curation, 2016
- Roles for Synonymous Codon Usage in Protein BiogenesisAnnual Review of Biophysics, 2015
- Timing Is Everything: Unifying Codon Translation Rates and Nascent Proteome BehaviorJournal of the American Chemical Society, 2014
- Expanding Anfinsen’s Principle: Contributions of Synonymous Codon Selection to Rational Protein DesignJournal of the American Chemical Society, 2014
- CD-HIT: accelerated for clustering the next-generation sequencing dataBioinformatics, 2012
- PaxDb, a Database of Protein Abundance Averages Across All Three Domains of LifeMolecular & Cellular Proteomics, 2012
- Synonymous codon usage influences the local protein structure observedNucleic Acids Research, 2010
- UniRef: comprehensive and non-redundant UniProt reference clustersBioinformatics, 2007
- Codon usage tabulated from international DNA sequence databases: status for the year 2000Nucleic Acids Research, 2000
- The codon adaptation index-a measure of directional synonymous codon usage bias, and its potential applicationsNucleic Acids Research, 1987