Dana Lee Cortade

Generative models are increasingly used for protein design, but the lack of standardized evaluation frameworks limits comparison across model classes and hinders translation to experimental success. Here, we introduce a unified sampling and benchmarking framework that enables controlled sequence generation across alignment, protein language, and structure-based models, and apply it to Tobacco etch virus (TEV) protease. Across hundreds of thousands of designed sequences, different models explore distinct regions of sequence space with no clear computational selection metrics to assess enzymatic function. Experimental evaluation reveals large differences in functional outcomes, ranging from non-functional variants to sequences with 9-fold higher activity than wildtype. Machine learning-designed libraries achieve a 39.32% hit rate (percentage of variants matching or exceeding wildtype activity) compared to 6.06% for an error-prone PCR baseline. Structure-based models perform best overall, with hit rates of 74.4% and 66.8% for ESM-IF1 and ProteinMPNN, respectively. Commonly used selection metrics do not strongly correlate with experimental activity, highlighting a gap between in silico evaluation and enzyme function. Together, these results establish a generalizable framework for benchmarking generative protein models and demonstrate the necessity of experimental validation for guiding model development and sequence prioritization.

Predicting any protein’s function from its sequence alone would be a significant breakthrough in molecular biology. Although machine learning approaches have sought to tackle this, their limited generalizability reflects the absence of sufficiently large, open, diverse, and unified datasets. To address this data gap, we developed a high-throughput experimental platform called GROQ-seq ( Gro wth-based Q uantitative Seq uencing). In GROQ-seq, a protein’s function can be linked to a sequencing-based readout that enables scalable characterization of large variant libraries in Escherichia coli . Here, we present pilot datasets demonstrating its performance across three distinct protein function classes: transcription factors, polymerases, and proteases. The objective of this report is to present the datasets and to provide users with a clear and transparent characterization of their properties, including both the strengths and limitations.

High-throughput functional assays are increasingly used to generate large-scale protein function datasets for protein engineering and machine learning applications. However, the utility of such datasets depends on the reproducibility of the underlying measurements. Here we report reproducible, quantitative measurements of protein sequence-to-function data at scale across two facilities. We analyze GROQ-seq ( G rowth-based Q uantitative Seq uencing) measurements of three bacterial transcription factors. Independent barcode measurements of the same sequence produce highly consistent functional estimates, demonstrating strong biological reproducibility (across all transcription factors the mean Root Mean Square Deviation [RMSD] ≈ 0.53 and mean Spearman ≈ 0.63). We also compared experiments performed at two facilities using a shared protocol, but with differing levels of automation and system integration. We observe strong agreement between measurements taken at the two sites (mean RMSD ≈ 0.41 and mean Spearman ≈ 0.730). Orthogonal tests further support this agreement: a classifier trained to distinguish data by site performs near random (AUC = 0.559), and top-ranking variants show strong statistical overlap between experiments. Together, these results demonstrate that GROQ-seq enables reproducible, scalable measurement of protein function suitable for large aggregated datasets.

The grand challenge of protein engineering is the development of computational models to characterize and generate protein sequences for arbitrary functions. Progress is limited by lack of 1) benchmarking opportunities, 2) large protein function datasets, and 3) access to experimental protein characterization. We introduce the Protein Engineering Tournament—a fully-remote competition designed to foster the development and evaluation of computational approaches in protein engineering. The tournament consists of an in silico round, predicting biophysical properties from protein sequences, followed by an in vitro round where novel protein sequences are designed, expressed and characterized using automated methods. Upon completion, all datasets, experimental protocols, and methods are made publicly available. We detail the structure and outcomes of a pilot Tournament involving seven protein design teams, powered by six multi-objective datasets, with experimental characterization by our partner, International Flavors and Fragrances. Forthcoming Protein Engineering Tournaments aim to mobilize the scientific community towards transparent evaluation of progress in the field. Graphical

The grand challenge of protein engineering is the development of computational models that can characterize and generate protein sequences for any arbitrary function. However, progress today is limited by lack of 1) benchmarks with which to compare computational techniques, 2) large datasets of protein function, and 3) democratized access to experimental protein characterization. Here, we introduce the Protein Engineering Tournament, a fully-remote, biennial competition for the development and benchmarking of computational methods in protein engineering. The tournament consists of two rounds: a first in silico round, where participants use computational models to predict biophysical properties for a set of protein sequences, and a second in vitro round, where participants are challenged to design new protein sequences, which are experimentally measured with open-source, automated methods to determine a winner. At the Tournament's conclusion, the experimental protocols and all collected data will be open-sourced for continued benchmarking and advancement of computational models. We hope the Protein Engineering Tournament will provide a transparent platform with which to evaluate progress in this field and mobilize the scientific community to conquer the grand challenge of computational protein engineering.

Genetic testing is considered a cornerstone of the precision medicine paradigm. Genotyping of single nucleotide polymorphisms (SNPs) has been shown to provide insights into several important issues, including therapy selection and drug responsiveness. However, a scarcity of widely deployable and cost-effective genotyping tools has limited the integration of precision medicine into routine clinical practice. The objective of our work was to develop a portable, cost-effective, and automated platform that performs SNP genotyping at the point-of-care (POC). Using recombinase polymerase amplification (RPA) and giant magnetoresistive (GMR) nanosensors, we present a highly automated and multiplexed point-of-care platform that utilizes direct saliva for the qualitative genotyping of four SNPs (rs4633, rs4680, rs4818, rs6269) along the catechol-O-methyltransferase gene (COMT), which is associated with the modulation of pain sensitivity and perioperative opioid use. Using this approach, we successfully amplify, detect, and genotype all four of the SNPs, demonstrating 100% accordance between the experimental results obtained using the automated RPA and GMR genotyping assay and the results obtained using a COMT PCR genotyping assay that was formerly validated using pyrosequencing. This automated, portable, and multiplexed RPA and GMR assay shows great promise as a solution for SNP genotyping at the POC and reinforces the broad applications of magnetic nanotechnology in biomedicine.

Morphine is an analgesic narcotic drug with a high risk for addiction. Testing standards require urine samples processed by central laboratories, leading to delayed results. Giant magnetoresistive sensors coupled with super paramagnetic nanoparticles offer a quantitative, sensitive, and low-cost solution for morphine detection in saliva at the point of care. The smartphone-controlled platform transmits results via Bluetooth, takes 35 minutes from sample to result, and delivers a lower limit of detection of 3.78 ng/mL and a dynamic range up to 500 ng/mL. The affordable and multiplexable GMR nanosensor platform enables this technology to be widely utilized for workplace drug testing.