Garyk Brixi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Garyk Brixi has a total of 147 co-authors. Here are the top co-authors by number of shared publications:
- Brian L. Hie
Arc Institute, Palo Alto, CA, USA
7 shared publications - David B. Li
Department of Bioengineering, Stanford University, Stanford, CA, USA
5 shared publications - Samuel H. King
Department of Bioengineering, Stanford University, Stanford, CA, USA.
5 shared publications - Aditi T. Merchant
Arc Institute, Palo Alto, CA, USA
4 shared publications - Claudia L. Driscoll
Department of Pharmacology, University of Cambridge, Tennis Court Road, Cambridge, CB2 1PD, UK
4 shared publications - Max E. Wilkinson
Structural Biology Program, Sloan Kettering Institute, Memorial Sloan Kettering Cancer Center, New York, NY, USA
3 shared publications
Latest Publications
- Open Access
Efficient biological design via genomic in-context learning
In-context learning (ICL), the ability to perform new tasks from a few examples supplied in a prompt, is a defining emergent capability of large language models. Here, we show that ICL emerges in the genome language model Evo 2 and enables biological design. With as few as four examples in the prompt and no additional training, Evo 2 generates realistic metal-binding proteins, de novo antibodies, protein-RNA systems, and entire bacteriophage genomes. For ΦX174-like bacteriophage design, ICL surpasses previously fine-tuned models on computational measures of design quality and diversity. Beyond ΦX174, conditioning the model with different phage prompts enables us to efficiently design MS2 and M13 genomes in silico without fine-tuning. Iteratively updating the prompt with newly generated phage genomes, which we refer to as genomic chain-of-thought, further improves the diversity of designs while maintaining quality metrics. Paired with an inference engine that accelerates generation up to 35-fold and extends practicable context to a megabase, these results establish ICL as a versatile and efficient paradigm for biological sequence design.
Open AccessEfficient biological design via genomic in-context learning
In-context learning (ICL), the ability to perform new tasks from a few examples supplied in a prompt, is a defining emergent capability of large language models. Here, we show that ICL emerges in the genome language model Evo 2 and enables biological design. With as few as four examples in the prompt and no additional training, Evo 2 generates realistic metal-binding proteins, de novo antibodies, protein-RNA systems, and entire bacteriophage genomes. For ΦX174-like bacteriophage design, ICL surpasses previously fine-tuned models on computational measures of design quality and diversity. Beyond ΦX174, conditioning the model with different phage prompts enables us to efficiently design MS2 and M13 genomes in silico without fine-tuning. Iteratively updating the prompt with newly generated phage genomes, which we refer to as genomic chain-of-thought, further improves the diversity of designs while maintaining quality metrics. Paired with an inference engine that accelerates generation up to 35-fold and extends practicable context to a megabase, these results establish ICL as a versatile and efficient paradigm for biological sequence design.
- Open Access
Coevolutionary mining of prokaryotic non-coding elements with a genome language model
Microbial genomes encode non-coding RNAs (ncRNAs) and nucleic-acid-interacting proteins essential to biology and biotechnology. However, discovery of these systems remains protein-centric and based on similarity to known sequences. Here, we introduce Minerva, a framework for coevolutionary mining that uses genome language models to predict pairwise interactions directly from sequence. Minerva enables fast, alignment-free prediction of base pairing, monomeric protein contacts, and repetitive sequence motifs. We recover known systems such as CRISPR, detect open-reading-frame signatures at the DNA level without supervision, and predict hundreds of unannotated regions per bacterial genome containing multiple hairpins or repetitive motifs. In Pseudomonadota, we find that the widespread TwoAYGGAY structured RNA family carries large secondary-structure extensions and is often flanked by repetitive motifs. In prophages, we discover that Unknown Group 27 reverse transcriptase systems encode variable arrays of structurally conserved yet sequence-diverse ncRNAs that template complementary DNA hairpin products. Coevolutionary mining with Minerva enables efficient biological discovery across microbial genomes and metagenomes.
Open AccessCoevolutionary mining of prokaryotic non-coding elements with a genome language model
Microbial genomes encode non-coding RNAs (ncRNAs) and nucleic-acid-interacting proteins essential to biology and biotechnology. However, discovery of these systems remains protein-centric and based on similarity to known sequences. Here, we introduce Minerva, a framework for coevolutionary mining that uses genome language models to predict pairwise interactions directly from sequence. Minerva enables fast, alignment-free prediction of base pairing, monomeric protein contacts, and repetitive sequence motifs. We recover known systems such as CRISPR, detect open-reading-frame signatures at the DNA level without supervision, and predict hundreds of unannotated regions per bacterial genome containing multiple hairpins or repetitive motifs. In Pseudomonadota, we find that the widespread TwoAYGGAY structured RNA family carries large secondary-structure extensions and is often flanked by repetitive motifs. In prophages, we discover that Unknown Group 27 reverse transcriptase systems encode variable arrays of structurally conserved yet sequence-diverse ncRNAs that template complementary DNA hairpin products. Coevolutionary mining with Minerva enables efficient biological discovery across microbial genomes and metagenomes.
- Open Access
Forecasting viral antigenic evolution from phylogenetic trees
Forecasting the evolution of viral proteins can help inform pandemic preparedness and guide the development of interventions. Phylogenetic trees provide historical records of pathogen evolution, which are key for retrospective analysis, but using them to predict future genotypic changes remains challenging. We introduce antiGen, a machine learning model that learns from the evolutionary past of viruses to forecast the evolutionary future. antiGen achieves state-of-the-art performance in forecasting SARS-CoV-2 spike protein mutations according to a variety of metrics, including the prediction of mutations in the receptor binding domain, mutations never previously seen, and mutations with high prevalence. Most experimentally tested antiGen-forecasted spike variants retain pseudoviral infectivity in vitro, supporting their functional plausibility. Evaluations on surface proteins of influenza virus, respiratory syncytial virus, and dengue virus further demonstrate applicability across viruses with differing amounts of available sequencing data. Finally, we develop a model trained across 49 viral proteins from 27 viruses, which improves forecasting performance over models trained on individual protein phylogenies. Together, these results support learning from phylogenetic histories as an approach to model viral amino acid changes prospectively, with applications in epidemiology and therapeutic development.
Open AccessForecasting viral antigenic evolution from phylogenetic trees
Forecasting the evolution of viral proteins can help inform pandemic preparedness and guide the development of interventions. Phylogenetic trees provide historical records of pathogen evolution, which are key for retrospective analysis, but using them to predict future genotypic changes remains challenging. We introduce antiGen, a machine learning model that learns from the evolutionary past of viruses to forecast the evolutionary future. antiGen achieves state-of-the-art performance in forecasting SARS-CoV-2 spike protein mutations according to a variety of metrics, including the prediction of mutations in the receptor binding domain, mutations never previously seen, and mutations with high prevalence. Most experimentally tested antiGen-forecasted spike variants retain pseudoviral infectivity in vitro, supporting their functional plausibility. Evaluations on surface proteins of influenza virus, respiratory syncytial virus, and dengue virus further demonstrate applicability across viruses with differing amounts of available sequencing data. Finally, we develop a model trained across 49 viral proteins from 27 viruses, which improves forecasting performance over models trained on individual protein phylogenies. Together, these results support learning from phylogenetic histories as an approach to model viral amino acid changes prospectively, with applications in epidemiology and therapeutic development.
- Top Cited Papers
- 6 August 2026
- Published by American Association for the Advancement of Science (AAAS) in Science
- Vol. 393 (6811)
Generative design of bacteriophages with genome language models
Many important biological functions arise not from single genes but from complex interactions encoded by entire genomes. We report the first generative design of complete bacteriophage genomes using genome language models. We generated viable bacteriophages with target host tropism, using the phage ΦX174 as our design template. Experimental testing yielded 16 phages with diverse fitness profiles in laboratory conditions. Cryo–electron microscopy confirmed that a generated phage utilizes an evolutionarily distant DNA packaging protein in its capsid. A cocktail of generated phages rapidly overcomes ΦX174-resistant Escherichia coli strains, demonstrating a path toward artificial intelligence–generated phage therapies against rapidly evolving bacterial pathogens. This work provides a blueprint for the design of diverse synthetic bacteriophages and useful biological systems at the genome scale.
- 6 August 2026
- Published by American Association for the Advancement of Science (AAAS) in Science
- Vol. 393 (6811)
Top Cited PapersGenerative design of bacteriophages with genome language models
Many important biological functions arise not from single genes but from complex interactions encoded by entire genomes. We report the first generative design of complete bacteriophage genomes using genome language models. We generated viable bacteriophages with target host tropism, using the phage ΦX174 as our design template. Experimental testing yielded 16 phages with diverse fitness profiles in laboratory conditions. Cryo–electron microscopy confirmed that a generated phage utilizes an evolutionarily distant DNA packaging protein in its capsid. A cocktail of generated phages rapidly overcomes ΦX174-resistant Escherichia coli strains, demonstrating a path toward artificial intelligence–generated phage therapies against rapidly evolving bacterial pathogens. This work provides a blueprint for the design of diverse synthetic bacteriophages and useful biological systems at the genome scale.
- Top Cited PapersOpen Access
- 4 March 2026
- Published by Springer Nature in Nature
- Vol. 652 (8112) , 1-13
Genome modelling and design across all domains of life with Evo 2
All of life encodes information with DNA. Although tools for genome sequencing, synthesis and editing have transformed biological research, we still lack sufficient understanding of the immense complexity encoded by genomes to predict the effects of many classes of genomic changes or to intelligently compose new biological systems. Artificial intelligence models that learn information from genomic sequences across diverse organisms have increasingly advanced prediction and design capabilities1,2. Here we introduce Evo 2, a biological foundation model trained on 9 trillion DNA base pairs from a highly curated genomic atlas spanning all domains of life to have a 1 million token context window with single-nucleotide resolution. Evo 2 learns to accurately predict the functional impacts of genetic variation—from noncoding pathogenic mutations to clinically significant BRCA1 variants—without task-specific fine-tuning. Mechanistic interpretability analyses reveal that Evo 2 learns representations associated with biological features, including exon–intron boundaries, transcription factor binding sites, protein structural elements and prophage genomic regions. The generative abilities of Evo 2 produce mitochondrial, prokaryotic and eukaryotic sequences at genome scale with greater naturalness and coherence than previous methods. Evo 2 also generates experimentally validated chromatin accessibility patterns when guided by predictive models3,4 and inference-time search. We have made Evo 2 fully open, including model parameters, training code5, inference code and the OpenGenome2 dataset, to accelerate the exploration and design of biological complexity.
- 4 March 2026
- Published by Springer Nature in Nature
- Vol. 652 (8112) , 1-13
Top Cited PapersOpen AccessGenome modelling and design across all domains of life with Evo 2
All of life encodes information with DNA. Although tools for genome sequencing, synthesis and editing have transformed biological research, we still lack sufficient understanding of the immense complexity encoded by genomes to predict the effects of many classes of genomic changes or to intelligently compose new biological systems. Artificial intelligence models that learn information from genomic sequences across diverse organisms have increasingly advanced prediction and design capabilities1,2. Here we introduce Evo 2, a biological foundation model trained on 9 trillion DNA base pairs from a highly curated genomic atlas spanning all domains of life to have a 1 million token context window with single-nucleotide resolution. Evo 2 learns to accurately predict the functional impacts of genetic variation—from noncoding pathogenic mutations to clinically significant BRCA1 variants—without task-specific fine-tuning. Mechanistic interpretability analyses reveal that Evo 2 learns representations associated with biological features, including exon–intron boundaries, transcription factor binding sites, protein structural elements and prophage genomic regions. The generative abilities of Evo 2 produce mitochondrial, prokaryotic and eukaryotic sequences at genome scale with greater naturalness and coherence than previous methods. Evo 2 also generates experimentally validated chromatin accessibility patterns when guided by predictive models3,4 and inference-time search. We have made Evo 2 fully open, including model parameters, training code5, inference code and the OpenGenome2 dataset, to accelerate the exploration and design of biological complexity.
- Top Cited PapersOpen Access
Generative design of novel bacteriophages with genome language models
Many important biological functions arise not from single genes, but from complex interactions encoded by entire genomes. Genome language models have emerged as a promising strategy for designing biological systems, but their ability to generate functional sequences at the scale of whole genomes has remained untested. Here, we report the first generative design of viable bacteriophage genomes. We leveraged frontier genome language models, Evo 1 and Evo 2, to generate whole-genome sequences with realistic genetic architectures and desirable host tropism, using the lytic phage ΦX174 as our design template. Experimental testing of AI-generated genomes yielded 16 viable phages with substantial evolutionary novelty. Cryo-electron microscopy revealed that one of the generated phages utilizes an evolutionarily distant DNA packaging protein within its capsid. Multiple phages demonstrate higher fitness than ΦX174 in growth competitions and in their lysis kinetics. A cocktail of the generated phages rapidly overcomes ΦX174-resistance in three E. coli strains, demonstrating the potential utility of our approach for designing phage therapies against rapidly evolving bacterial pathogens. This work provides a blueprint for the design of diverse synthetic bacteriophages and, more broadly, lays a foundation for the generative design of useful living systems at the genome scale.
Top Cited PapersOpen AccessGenerative design of novel bacteriophages with genome language models
Many important biological functions arise not from single genes, but from complex interactions encoded by entire genomes. Genome language models have emerged as a promising strategy for designing biological systems, but their ability to generate functional sequences at the scale of whole genomes has remained untested. Here, we report the first generative design of viable bacteriophage genomes. We leveraged frontier genome language models, Evo 1 and Evo 2, to generate whole-genome sequences with realistic genetic architectures and desirable host tropism, using the lytic phage ΦX174 as our design template. Experimental testing of AI-generated genomes yielded 16 viable phages with substantial evolutionary novelty. Cryo-electron microscopy revealed that one of the generated phages utilizes an evolutionarily distant DNA packaging protein within its capsid. Multiple phages demonstrate higher fitness than ΦX174 in growth competitions and in their lysis kinetics. A cocktail of the generated phages rapidly overcomes ΦX174-resistance in three E. coli strains, demonstrating the potential utility of our approach for designing phage therapies against rapidly evolving bacterial pathogens. This work provides a blueprint for the design of diverse synthetic bacteriophages and, more broadly, lays a foundation for the generative design of useful living systems at the genome scale.
- Open Access
The Dayhoff Atlas: scaling sequence diversity for improved protein generation
Modern biology is powered by the organization of biological information, a framework pioneered in 1965 by Margaret Dayhoff's Atlas of Protein Sequence and Structure. Databases descended from this common ancestor power computational methods that have revolutionized our ability to understand and design proteins. We introduce the Dayhoff Atlas, a centralized collection of both protein sequence data and generative protein language models, as a modern-day resource for protein biology in the age of AI. We combined and reclustered metagenomic sequences with UniRef100 to create GigaRef, which includes 3.34 billion protein sequences across 1.70 billion clusters and provides the largest open dataset of natural proteins to date. To fuse the richness of protein structure with the scalability of sequence space, we generated structure-based synthetic data, producing BackboneRef, a first-in-class dataset of 46 million synthetic protein sequences predicted from 240,811 de novo designed backbones. Using these data and evolutionary sequence alignments, we trained the Dayhoff family of protein language models, including the first model that combines single proteins and sets of evolutionarily-related sequences at scale. Operating only in amino acid sequence space, the Dayhoff models can natively predict mutation effects on fitness, scaffold structural motifs, and perform guided generation of new proteins within a specific family. Learning from metagenomic and structure-based synthetic data from the Dayhoff Atlas increased the cellular expression rates of generated proteins, highlighting the real-world value of expanding the scale, diversity, and novelty of protein sequence data. We release the Dayhoff Atlas, including all datasets, code, and models, under a permissive license to continue to bring the power of computation to protein biology and design.
Open AccessThe Dayhoff Atlas: scaling sequence diversity for improved protein generation
Modern biology is powered by the organization of biological information, a framework pioneered in 1965 by Margaret Dayhoff's Atlas of Protein Sequence and Structure. Databases descended from this common ancestor power computational methods that have revolutionized our ability to understand and design proteins. We introduce the Dayhoff Atlas, a centralized collection of both protein sequence data and generative protein language models, as a modern-day resource for protein biology in the age of AI. We combined and reclustered metagenomic sequences with UniRef100 to create GigaRef, which includes 3.34 billion protein sequences across 1.70 billion clusters and provides the largest open dataset of natural proteins to date. To fuse the richness of protein structure with the scalability of sequence space, we generated structure-based synthetic data, producing BackboneRef, a first-in-class dataset of 46 million synthetic protein sequences predicted from 240,811 de novo designed backbones. Using these data and evolutionary sequence alignments, we trained the Dayhoff family of protein language models, including the first model that combines single proteins and sets of evolutionarily-related sequences at scale. Operating only in amino acid sequence space, the Dayhoff models can natively predict mutation effects on fitness, scaffold structural motifs, and perform guided generation of new proteins within a specific family. Learning from metagenomic and structure-based synthetic data from the Dayhoff Atlas increased the cellular expression rates of generated proteins, highlighting the real-world value of expanding the scale, diversity, and novelty of protein sequence data. We release the Dayhoff Atlas, including all datasets, code, and models, under a permissive license to continue to bring the power of computation to protein biology and design.
- Top Cited Papers
- 15 November 2024
- Published by American Association for the Advancement of Science (AAAS) in Science
- Vol. 386 (6723) , eado9336
Sequence modeling and design from molecular to genome scale with Evo
The genome is a sequence that encodes the DNA, RNA, and proteins that orchestrate an organism’s function. We present Evo, a long-context genomic foundation model with a frontier architecture trained on millions of prokaryotic and phage genomes, and report scaling laws on DNA to complement observations in language and vision. Evo generalizes across DNA, RNA, and proteins, enabling zero-shot function prediction competitive with domain-specific language models and the generation of functional CRISPR-Cas and transposon systems, representing the first examples of protein-RNA and protein-DNA codesign with a language model. Evo also learns how small mutations affect whole-organism fitness and generates megabase-scale sequences with plausible genomic architecture. These prediction and generation capabilities span molecular to genomic scales of complexity, advancing our understanding and control of biology.
- 15 November 2024
- Published by American Association for the Advancement of Science (AAAS) in Science
- Vol. 386 (6723) , eado9336
Top Cited PapersSequence modeling and design from molecular to genome scale with Evo
The genome is a sequence that encodes the DNA, RNA, and proteins that orchestrate an organism’s function. We present Evo, a long-context genomic foundation model with a frontier architecture trained on millions of prokaryotic and phage genomes, and report scaling laws on DNA to complement observations in language and vision. Evo generalizes across DNA, RNA, and proteins, enabling zero-shot function prediction competitive with domain-specific language models and the generation of functional CRISPR-Cas and transposon systems, representing the first examples of protein-RNA and protein-DNA codesign with a language model. Evo also learns how small mutations affect whole-organism fitness and generates megabase-scale sequences with plausible genomic architecture. These prediction and generation capabilities span molecular to genomic scales of complexity, advancing our understanding and control of biology.
- Top Cited Papers
- 25 September 2024
- Published by Springer Nature in Nature
- Vol. 638 (8051) , 787-795
Transferrin receptor targeting chimeras for membrane protein degradation
Cancer cells require high levels of iron for rapid proliferation, leading to significant upregulation of cell-surface transferrin receptor 1 (TfR1), which mediates iron uptake by binding to the iron-carrying protein transferrin1,2,3. Leveraging this phenomenon and the fast endocytosis rate of TfR1 (refs. 4,5), we developed transferrin receptor targeting chimeras (TransTACs), a heterobispecific antibody modality for membrane protein degradation. TransTACs are engineered to drive rapid co-internalization of a target protein of interest and TfR1 from the cell surface, and to enable target protein entry into the lysosomal degradation pathway. We show that TransTACs can efficiently degrade a diverse range of single-pass, multi-pass, native or synthetic membrane proteins, including epidermal growth factor receptor, programmed cell death 1 ligand 1, cluster of differentiation 20 and chimeric antigen receptor. In example applications, TransTACs enabled the reversible control of human primary chimeric antigen receptor T cells and the targeting of drug-resistant epidermal growth factor receptor-driven lung cancer with the exon 19 deletion/T790M/C797S mutations in a mouse xenograft model. TransTACs represent a promising new family of bifunctional antibodies for precise manipulation of membrane proteins and targeted cancer therapy.
- 25 September 2024
- Published by Springer Nature in Nature
- Vol. 638 (8051) , 787-795
Top Cited PapersTransferrin receptor targeting chimeras for membrane protein degradation
Cancer cells require high levels of iron for rapid proliferation, leading to significant upregulation of cell-surface transferrin receptor 1 (TfR1), which mediates iron uptake by binding to the iron-carrying protein transferrin1,2,3. Leveraging this phenomenon and the fast endocytosis rate of TfR1 (refs. 4,5), we developed transferrin receptor targeting chimeras (TransTACs), a heterobispecific antibody modality for membrane protein degradation. TransTACs are engineered to drive rapid co-internalization of a target protein of interest and TfR1 from the cell surface, and to enable target protein entry into the lysosomal degradation pathway. We show that TransTACs can efficiently degrade a diverse range of single-pass, multi-pass, native or synthetic membrane proteins, including epidermal growth factor receptor, programmed cell death 1 ligand 1, cluster of differentiation 20 and chimeric antigen receptor. In example applications, TransTACs enabled the reversible control of human primary chimeric antigen receptor T cells and the targeting of drug-resistant epidermal growth factor receptor-driven lung cancer with the exon 19 deletion/T790M/C797S mutations in a mouse xenograft model. TransTACs represent a promising new family of bifunctional antibodies for precise manipulation of membrane proteins and targeted cancer therapy.
- Open Access
Protein language models learn evolutionary statistics of interacting sequence motifs
Protein language models (pLMs) have emerged as potent tools for predicting and designing protein structure and function, and the degree to which these models fundamentally understand the inherent biophysics of protein structure stands as an open question. Motivated by a discovery that pLM-based structure predictors erroneously predict nonphysical structures for protein isoforms, we investigated the nature of sequence context needed for contact predictions in the pLM ESM-2. We demonstrate by use of a “categorical Jacobian” calculation that ESM-2 stores statistics of coevolving residues, analogously to simpler modelling approaches like Markov Random Fields and Multivariate Gaussian models. We further investigated how ESM-2 “stores” information needed to predict contacts by comparing sequence masking strategies, and found that providing local windows of sequence information allowed ESM-2 to best recover predicted contacts. This suggests that pLMs predict contacts by storing motifs of pairwise contacts. Our investigation highlights the limitations of current pLMs and underscores the importance of understanding the underlying mechanisms of these models. Significance Statement: Protein language models (pLMs) have exhibited remarkable capabilities in protein structure prediction and design. However, the extent to which they comprehend the intrinsic biophysics of protein structures remains uncertain. We present a suite of analyses that dissect how the flagship pLM ESM-2 predicts structure. Motivated by a consistent error of protein isoforms predicted as structured fragments, we developed a completely unsupervised method to uniformly evaluate any protein language model that allows for us to compare coevolutionary statistics to older linear models. We further identified t hat E SM-2 a ppears to have a precise context size that is needed to predict inter-residue contacts. Our study highlights the current limitations of pLMs and contributes to a deeper understanding of their underlying mechanisms, paving the way for more reliable protein structure predictions.
Open AccessProtein language models learn evolutionary statistics of interacting sequence motifs
Protein language models (pLMs) have emerged as potent tools for predicting and designing protein structure and function, and the degree to which these models fundamentally understand the inherent biophysics of protein structure stands as an open question. Motivated by a discovery that pLM-based structure predictors erroneously predict nonphysical structures for protein isoforms, we investigated the nature of sequence context needed for contact predictions in the pLM ESM-2. We demonstrate by use of a “categorical Jacobian” calculation that ESM-2 stores statistics of coevolving residues, analogously to simpler modelling approaches like Markov Random Fields and Multivariate Gaussian models. We further investigated how ESM-2 “stores” information needed to predict contacts by comparing sequence masking strategies, and found that providing local windows of sequence information allowed ESM-2 to best recover predicted contacts. This suggests that pLMs predict contacts by storing motifs of pairwise contacts. Our investigation highlights the limitations of current pLMs and underscores the importance of understanding the underlying mechanisms of these models. Significance Statement: Protein language models (pLMs) have exhibited remarkable capabilities in protein structure prediction and design. However, the extent to which they comprehend the intrinsic biophysics of protein structures remains uncertain. We present a suite of analyses that dissect how the flagship pLM ESM-2 predicts structure. Motivated by a consistent error of protein isoforms predicted as structured fragments, we developed a completely unsupervised method to uniformly evaluate any protein language model that allows for us to compare coevolutionary statistics to older linear models. We further identified t hat E SM-2 a ppears to have a precise context size that is needed to predict inter-residue contacts. Our study highlights the current limitations of pLMs and contributes to a deeper understanding of their underlying mechanisms, paving the way for more reliable protein structure predictions.