Sarah M. Burbach
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sarah M. Burbach has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Bryan Briney
Department of Immunology and Microbiology, The Scripps Research Institute, La Jolla, CA, USA.
5 shared publications Jonathan Hurtado
2 shared publications- Andrew B. Ward
Scripps Research Institute
1 shared publication - Charlotte DiBiase
The Scripps Research Institute
1 shared publication - Daniel Muli Muema
African Health Research Institute, Durban, South Africa
1 shared publication - Gabriel Avillion
Department of Immunology and Microbiology, the Scripps Research Institute, La Jolla, CA 92037, USA
1 shared publication
Latest Publications
- Open Access
Evaluating Expert Specialization in Mixture-of-Experts Antibody Language Models
A bstract: Antibody language models (AbLMs) show an impressive aptitude for learning antibody features, but tend to struggle learning the highly diverse, non-templated regions of antibodies. Existing AbLMs use dense architectures, where all model parameters attend to each amino acid token. We hypothesized that the modular nature of antibodies could benefit from a sparse mixture-of-experts (MoE) architecture, allowing specific parameters (referred to as ‘experts’) to specialize in distinct antibody features. While MoE architectures are widely adopted and optimized in natural language processing domains, they are less common in biological modeling. To this end, we assess existing MoE routing strategies and find that token-choice routing strategies outperform expert-choice routing, presumably due to their specialization in CDRH3 residues. We further optimized the token-choice router for AbLMs, by minimizing the routing of padding tokens to enable pre-training with varying sequence lengths. Finally, we show that a large-scale baseline antibody language model with a Top-2 MoE architecture (BALM-MoE), trained on a mixture of unpaired and paired antibody sequences, outperforms its dense counterpart with the same number of active parameters.
Open AccessEvaluating Expert Specialization in Mixture-of-Experts Antibody Language Models
A bstract: Antibody language models (AbLMs) show an impressive aptitude for learning antibody features, but tend to struggle learning the highly diverse, non-templated regions of antibodies. Existing AbLMs use dense architectures, where all model parameters attend to each amino acid token. We hypothesized that the modular nature of antibodies could benefit from a sparse mixture-of-experts (MoE) architecture, allowing specific parameters (referred to as ‘experts’) to specialize in distinct antibody features. While MoE architectures are widely adopted and optimized in natural language processing domains, they are less common in biological modeling. To this end, we assess existing MoE routing strategies and find that token-choice routing strategies outperform expert-choice routing, presumably due to their specialization in CDRH3 residues. We further optimized the token-choice router for AbLMs, by minimizing the routing of padding tokens to enable pre-training with varying sequence lengths. Finally, we show that a large-scale baseline antibody language model with a Top-2 MoE architecture (BALM-MoE), trained on a mixture of unpaired and paired antibody sequences, outperforms its dense counterpart with the same number of active parameters.
- Open Access
- 11 September 2025
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 21 (9) , e1013473
A curriculum learning approach to training antibody language models
There is growing interest in pre-training antibody language models (AbLMs) with a mixture of unpaired and natively paired sequences, seeking to combine the proven benefits of training with natively paired sequences with the massive scale of unpaired antibody sequence datasets. However, given the novelty of this strategy, the field lacks a systematic evaluation of data processing methods and training strategies that maximize the benefits of mixed training data while accommodating the significant imbalance in the size of existing paired and unpaired datasets. Here, we introduce a method of curriculum learning for AbLMs, which facilitates a gradual transition from unpaired to paired sequences during training. We optimize this method and compare it to other data sampling strategies for AbLMs, including a constant mix and a fine-tuning approach. We observe that the curriculum and constant approaches show improved performance compared to the fine-tuning approach in large-scale models, likely due to their ability to prevent catastrophic forgetting and slow overfitting. Finally, we show that a 650M-parameter curriculum model, CurrAb, outperforms existing mixed AbLMs in downstream residue prediction and classification tasks.
- 11 September 2025
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 21 (9) , e1013473
Open AccessA curriculum learning approach to training antibody language models
There is growing interest in pre-training antibody language models (AbLMs) with a mixture of unpaired and natively paired sequences, seeking to combine the proven benefits of training with natively paired sequences with the massive scale of unpaired antibody sequence datasets. However, given the novelty of this strategy, the field lacks a systematic evaluation of data processing methods and training strategies that maximize the benefits of mixed training data while accommodating the significant imbalance in the size of existing paired and unpaired datasets. Here, we introduce a method of curriculum learning for AbLMs, which facilitates a gradual transition from unpaired to paired sequences during training. We optimize this method and compare it to other data sampling strategies for AbLMs, including a constant mix and a fine-tuning approach. We observe that the curriculum and constant approaches show improved performance compared to the fine-tuning approach in large-scale models, likely due to their ability to prevent catastrophic forgetting and slow overfitting. Finally, we show that a 650M-parameter curriculum model, CurrAb, outperforms existing mixed AbLMs in downstream residue prediction and classification tasks.
- Open Access
Data-optimal scaling of paired antibody language models
Scaling laws for large language models in natural language domains are typically derived under the assumption that performance is primarily compute-constrained. In contrast, antibody language models (AbLMs) trained on paired sequences are primarily data-limited, thus requiring different considerations. To explore how model size and data scale affect AbLM performance, we trained 15 AbLMs across all pairwise combinations of five model sizes and three training data sizes. From these experiments, we derive an AbLM-specific scaling law and estimate that training a data-optimal AbLM equivalent of the highly performant 650M-parameter ESM-2 protein language model would require ∼5.5 million paired antibody sequences. Evaluation on multiple downstream classification tasks revealed that significant performance gains emerged only with sufficiently large model size, suggesting that in data-limited domains, improved performance depends jointly on both model scale and data volume.
Open AccessData-optimal scaling of paired antibody language models
Scaling laws for large language models in natural language domains are typically derived under the assumption that performance is primarily compute-constrained. In contrast, antibody language models (AbLMs) trained on paired sequences are primarily data-limited, thus requiring different considerations. To explore how model size and data scale affect AbLM performance, we trained 15 AbLMs across all pairwise combinations of five model sizes and three training data sizes. From these experiments, we derive an AbLM-specific scaling law and estimate that training a data-optimal AbLM equivalent of the highly performant 650M-parameter ESM-2 protein language model would require ∼5.5 million paired antibody sequences. Evaluation on multiple downstream classification tasks revealed that significant performance gains emerged only with sufficiently large model size, suggesting that in data-limited domains, improved performance depends jointly on both model scale and data volume.
- Open Access
A curriculum learning approach to training antibody language models
There is growing interest in pre-training antibody language models ( AbLMs ) with a mixture of unpaired and natively paired sequences, seeking to combine the proven benefits of training with natively paired sequences with the massive scale of unpaired antibody sequence datasets. However, given the novelty of this strategy, the field lacks a systematic evaluation of data processing methods and training strategies that maximize the benefits of mixed training data while accommodating the significant imbalance in the size of existing paired and unpaired datasets. Here we introduce a method of curriculum learning for AbLMs, which facilitates a gradual transition from unpaired to paired sequences during training. We optimize this method and show that a 650M-parameter curriculum model, CurrAb, outperforms existing mixed AbLMs in downstream classification tasks.
Open AccessA curriculum learning approach to training antibody language models
There is growing interest in pre-training antibody language models ( AbLMs ) with a mixture of unpaired and natively paired sequences, seeking to combine the proven benefits of training with natively paired sequences with the massive scale of unpaired antibody sequence datasets. However, given the novelty of this strategy, the field lacks a systematic evaluation of data processing methods and training strategies that maximize the benefits of mixed training data while accommodating the significant imbalance in the size of existing paired and unpaired datasets. Here we introduce a method of curriculum learning for AbLMs, which facilitates a gradual transition from unpaired to paired sequences during training. We optimize this method and show that a 650M-parameter curriculum model, CurrAb, outperforms existing mixed AbLMs in downstream classification tasks.
- Open Access
Conformational ensemble-based framework enables rapid development of Lassa virus vaccine candidates
Lassa virus (LASV), an arenavirus endemic to West Africa, poses a significant public health threat due to its high pathogenicity and expanding geographic risk zone. LASV glycoprotein complex (GPC) is the only known target of neutralizing antibodies, but its inherent metastability and conformational flexibility have hindered the development of GPC-based vaccines. We employed a variant of AlphaFold2 (AF2), called subsampled AF2, to generate diverse structures of LASV GPC that capture an array of potential conformational states using MSA subsampling and dropout layers. Conformational ensembles identified several metamorphic domains—areas of significant conformational flexibility—that could be targeted to stabilize the GPC in its immunogenic prefusion state. ProteinMPNN was then used to redesign GPC sequences to minimize the mobility of metamorphic domains. These redesigned sequences were further filtered using subsampled AF2, leading to the identification of promising GPC variants for further testing. A small library of redesigned GPC sequences was experimentally validated and showed significantly increased protein yields compared to controls. Antigenic profiles indicated these variants preserved essential epitopes for effective immune response, suggesting their potential for broad protective efficacy. Our results demonstrate that AI-driven approaches can predict the conformational landscape of complex pathogens. This knowledge can be used to stabilize viral proteins, such as LASV GPC, in their prefusion conformation, optimizing them for stability and expression, and offering a streamlined framework for vaccine design. Our deep learning / machine learning enabled framework contributes to global efforts to combat LASV and has broader implications for vaccine design and pandemic preparedness.
Open AccessConformational ensemble-based framework enables rapid development of Lassa virus vaccine candidates
Lassa virus (LASV), an arenavirus endemic to West Africa, poses a significant public health threat due to its high pathogenicity and expanding geographic risk zone. LASV glycoprotein complex (GPC) is the only known target of neutralizing antibodies, but its inherent metastability and conformational flexibility have hindered the development of GPC-based vaccines. We employed a variant of AlphaFold2 (AF2), called subsampled AF2, to generate diverse structures of LASV GPC that capture an array of potential conformational states using MSA subsampling and dropout layers. Conformational ensembles identified several metamorphic domains—areas of significant conformational flexibility—that could be targeted to stabilize the GPC in its immunogenic prefusion state. ProteinMPNN was then used to redesign GPC sequences to minimize the mobility of metamorphic domains. These redesigned sequences were further filtered using subsampled AF2, leading to the identification of promising GPC variants for further testing. A small library of redesigned GPC sequences was experimentally validated and showed significantly increased protein yields compared to controls. Antigenic profiles indicated these variants preserved essential epitopes for effective immune response, suggesting their potential for broad protective efficacy. Our results demonstrate that AI-driven approaches can predict the conformational landscape of complex pathogens. This knowledge can be used to stabilize viral proteins, such as LASV GPC, in their prefusion conformation, optimizing them for stability and expression, and offering a streamlined framework for vaccine design. Our deep learning / machine learning enabled framework contributes to global efforts to combat LASV and has broader implications for vaccine design and pandemic preparedness.