SBSarah M. Burbach

Sarah M. Burbach

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Sarah M. Burbach has a total of 18 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Evaluating Expert Specialization in Mixture-of-Experts Antibody Language Models

    by Sarah M. Burbach, Simone Spandau, Jonathan Hurtado, Bryan Briney

    A bstract: Antibody language models (AbLMs) show an impressive aptitude for learning antibody features, but tend to struggle learning the highly diverse, non-templated regions of antibodies. Existing AbLMs use dense architectures, where all model parameters attend to each amino acid token. We hypothesized that the modular nature of antibodies could benefit from a sparse mixture-of-experts (MoE) architecture, allowing specific parameters (referred to as ‘experts’) to specialize in distinct antibody features. While MoE architectures are widely adopted and optimized in natural language processing domains, they are less common in biological modeling. To this end, we assess existing MoE routing strategies and find that token-choice routing strategies outperform expert-choice routing, presumably due to their specialization in CDRH3 residues. We further optimized the token-choice router for AbLMs, by minimizing the routing of padding tokens to enable pre-training with varying sequence lengths. Finally, we show that a large-scale baseline antibody language model with a Top-2 MoE architecture (BALM-MoE), trained on a mixture of unpaired and paired antibody sequences, outperforms its dense counterpart with the same number of active parameters.

  • Open Access

    A curriculum learning approach to training antibody language models

    by Sarah M. Burbach, Bryan Briney

    There is growing interest in pre-training antibody language models (AbLMs) with a mixture of unpaired and natively paired sequences, seeking to combine the proven benefits of training with natively paired sequences with the massive scale of unpaired antibody sequence datasets. However, given the novelty of this strategy, the field lacks a systematic evaluation of data processing methods and training strategies that maximize the benefits of mixed training data while accommodating the significant imbalance in the size of existing paired and unpaired datasets. Here, we introduce a method of curriculum learning for AbLMs, which facilitates a gradual transition from unpaired to paired sequences during training. We optimize this method and compare it to other data sampling strategies for AbLMs, including a constant mix and a fine-tuning approach. We observe that the curriculum and constant approaches show improved performance compared to the fine-tuning approach in large-scale models, likely due to their ability to prevent catastrophic forgetting and slow overfitting. Finally, we show that a 650M-parameter curriculum model, CurrAb, outperforms existing mixed AbLMs in downstream residue prediction and classification tasks.


  • Open Access

    Data-optimal scaling of paired antibody language models

    by Mahdi Shafiei Neyestanak, Sarah M. Burbach, Karenna Ng, Praneeth Gangavarapu, Jonathan Hurtado, Judie Magura, Nasreen Ismail, Daniel Muema, Thumbi Ndung’u, Andrew B. Ward, et al.

    Scaling laws for large language models in natural language domains are typically derived under the assumption that performance is primarily compute-constrained. In contrast, antibody language models (AbLMs) trained on paired sequences are primarily data-limited, thus requiring different considerations. To explore how model size and data scale affect AbLM performance, we trained 15 AbLMs across all pairwise combinations of five model sizes and three training data sizes. From these experiments, we derive an AbLM-specific scaling law and estimate that training a data-optimal AbLM equivalent of the highly performant 650M-parameter ESM-2 protein language model would require ∼5.5 million paired antibody sequences. Evaluation on multiple downstream classification tasks revealed that significant performance gains emerged only with sufficiently large model size, suggesting that in data-limited domains, improved performance depends jointly on both model scale and data volume.


  • Open Access

    A curriculum learning approach to training antibody language models

    by Sarah M. Burbach, Bryan Briney

    There is growing interest in pre-training antibody language models ( AbLMs ) with a mixture of unpaired and natively paired sequences, seeking to combine the proven benefits of training with natively paired sequences with the massive scale of unpaired antibody sequence datasets. However, given the novelty of this strategy, the field lacks a systematic evaluation of data processing methods and training strategies that maximize the benefits of mixed training data while accommodating the significant imbalance in the size of existing paired and unpaired datasets. Here we introduce a method of curriculum learning for AbLMs, which facilitates a gradual transition from unpaired to paired sequences during training. We optimize this method and show that a 650M-parameter curriculum model, CurrAb, outperforms existing mixed AbLMs in downstream classification tasks.


  • Open Access

    Conformational ensemble-based framework enables rapid development of Lassa virus vaccine candidates

    by Nitesh Mishra, Gabriel Avillion, Sean Callaghan, Charlotte DiBiase, Jonathan Hurtado, Nathan Liendo, Sarah Burbach, Terrence Messmer, Bryan Briney

    Lassa virus (LASV), an arenavirus endemic to West Africa, poses a significant public health threat due to its high pathogenicity and expanding geographic risk zone. LASV glycoprotein complex (GPC) is the only known target of neutralizing antibodies, but its inherent metastability and conformational flexibility have hindered the development of GPC-based vaccines. We employed a variant of AlphaFold2 (AF2), called subsampled AF2, to generate diverse structures of LASV GPC that capture an array of potential conformational states using MSA subsampling and dropout layers. Conformational ensembles identified several metamorphic domains—areas of significant conformational flexibility—that could be targeted to stabilize the GPC in its immunogenic prefusion state. ProteinMPNN was then used to redesign GPC sequences to minimize the mobility of metamorphic domains. These redesigned sequences were further filtered using subsampled AF2, leading to the identification of promising GPC variants for further testing. A small library of redesigned GPC sequences was experimentally validated and showed significantly increased protein yields compared to controls. Antigenic profiles indicated these variants preserved essential epitopes for effective immune response, suggesting their potential for broad protective efficacy. Our results demonstrate that AI-driven approaches can predict the conformational landscape of complex pathogens. This knowledge can be used to stabilize viral proteins, such as LASV GPC, in their prefusion conformation, optimizing them for stability and expression, and offering a streamlined framework for vaccine design. Our deep learning / machine learning enabled framework contributes to global efforts to combat LASV and has broader implications for vaccine design and pandemic preparedness.