Dukka B. Kc

Post-translational modifications (PTMs) regulate protein function, making accurate residue-level PTM prediction essential for understanding cellular mechanisms and disease pathways. While decoder-only protein language models (PLMs) pretrained with the causal language modeling (CLM) objective have driven breakthroughs across various bioinformatics tasks, their potential for PTM prediction remains largely underexplored. CLM-based PLMs that rely on Byte-Pair Encoding (BPE) for tokenization, such as ProtGPT2, introduce intra-token label collision by merging multiple amino acids with conflicting labels into a single token, creating a major bottleneck for residue-level tasks. To overcome this, we propose TaHL-PTM (Target-Hooked Low-rank adaptation for PTM prediction), a novel framework that integrates target-hooked tokenization with site-directed discriminative LoRA fine-tuning. Target-hooked tokenization constrains tokenization around the candidate residue using dedicated marker tokens to eliminate intra-token label collision while preserving the surrounding sequence context, whereas the proposed discriminative objective repurposes the standard generative CLM objective for residue-level PTM classification by directly optimizing the separation between modified and unmodified sites. We benchmark TaHL-PTM across six distinct PTM tasks on ProtGPT2 and ProGen2 models. TaHL-PTM consistently improves MCC, with the largest gain of up to +0.11 for tyrosine phosphorylation (0.34 to 0.45), alongside improvements in F1, AUROC, and AUPR. Performance gains are more pronounced for collision-affected samples, validating the effectiveness of target-hooked tokenization, while consistent improvements across both BPE-based and per-residue-based causal PLMs demonstrate that the proposed framework generalizes across models with different pretraining tokenization schemes.

Protein functions often involve a dynamic interplay that covers a variety of molecular interactions that can be represented and analyzed in a 3-dimensional space. To this end, researchers have applied graph neural networks (GNNs) that effectively model such spaces as a promising methodology to predict protein functions. We discuss the graph-based representations of proteins that are applied to different prediction tasks, which include graphs at various levels of granularity: atomic, residue, and multi-scale. We also review various protein function prediction tools that rely on GNN architectures that learn representations from protein graphs, specifically in the context of the Gene Ontology prediction and protein–protein interaction prediction. GNN-based methods leverage the underlying structural knowledge and offer a promising future in improving the quality of the protein function predictions.

Advancements in sequencing technologies have resulted in a massive growth in the number of sequences available. Only a small fraction of the proteins in UniProtKB have been functionally annotated. Understanding the roles and studying the mechanisms of newly discovered proteins is one of the most important biological problems in the post-genomic era. To address the sequence-function gap many computational methods have been developed. This chapter reviews Multitask Learning (MTL)-based approaches for protein function prediction, highlighting its potential to enhance both predictive accuracy and computational efficiency in bioinformatics. MTL utilizes shared representations to leverage common information across related tasks, improving predictive performance. Key findings reveal that MTL improves predictive performance by integrating shared features across related tasks.

Inspired by the transformative success of large language models (LLMs) in natural language processing (NLP), numerous protein language models (PLMs) have recently emerged, revolutionizing the field of protein bioinformatics. PLMs have demonstrated remarkable achievements in representing proteins and designing new ones, capturing intrinsic structural and functional information trained on vast datasets of proteins, PLMs have demonstrated exceptional performance across a variety of bioinformatics tasks, including classification, function prediction, and de novo protein design. This chapter explores the evolution of PLMs, tracing their origins from NLP-based transformers and large language models (LLMs). A comprehensive summary of notable PLMs is presented, with a particular focus on encoder-only, encoder–decoder, and decoder-only architectures. Additionally, we delve into cutting-edge trends in PLM applications, such as fine-tuning methods, multimodal architectures, and the use of reduced alphabets. These innovations underscore the growing potential of PLMs to tackle complex biological problems and drive future breakthroughs in the field.

Post-translational modifications (PTMs) are vital regulators of protein function, influencing a myriad of cellular processes and disease mechanisms. Traditional experimental methods for PTM identification are both costly and labor-intensive, underlining the pressing need for efficient computational approaches. Early computational strategies predominantly relied on primary amino acid sequences and handcrafted features, which often lacked the contextual and structural understanding necessary for precise PTM site prediction. The emergence of transformer-based large language models (LLMs), particularly protein language models (pLMs), has revolutionized PTM prediction by producing context-aware embeddings that capture functional and structural intra-sequence dependencies. In this chapter, we provide a comprehensive review of recent advancements in leveraging LLMs (or, pLMs) for PTM site prediction, an important residue-level task in protein research. We identify emerging trends in the field, including the application of fine-tuning techniques, the integration of embeddings from multiple pLMs, and the incorporation of multiple modalities such as codon-aware embeddings, 3D structural data, and conventional representations. Additionally, we discuss tools that employ graph-based representations, the mamba architecture, and contrastive learning paradigms to further refine pLM-powered PTM site prediction models. We finally explore the interpretability and explainability aspects of the embeddings used in various tools. Despite the significant progress made, persistent limitations remain, and we outline these challenges while proposing directions for future research.

DNA-binding proteins (DBPs) play a crucial role in gene regulation, development, and environmental responses across plants, animals, and microorganisms. Existing DBP prediction methods are largely limited to sequence information, whether through handcrafted features or sequence-based protein language models (PLMs), overlooking structural cues critical to protein function. In addition, most existing tools are trained for general DBP predictions, which are often not accurate for plant-specific DBPs due to the unique structural and functional properties of plant proteins. Our work introduces PLM-DBPs, a deep learning framework that integrates both sequence-based and structure-aware representations to enhance DBP prediction in plants. We evaluated several state-of-the-art PLMs to extract high-dimensional protein representations and experimented with various fusion strategies to validate the complementary information between the various representations. Our final model, a fusion of sequence-based and structure-aware ANN models, achieves a notable improvement in predicting DBPs in plants outperforming previous state-of-the-art models. Although sequence-based PLMs already demonstrate strong performance in DBP prediction, our findings show that the integration of structural information further enhances predictive accuracy. This underscores the complementary nature of structural representations and establishes PLM-DBPs as a robust tool for advancing plant research and agricultural innovation. The proposed model and other resources are publicly available at https://github.com/suresh-pokharel/PLM-DBPs

Motivation: The mapping from codon to amino acid is surjective due to codon degeneracy, suggesting that codon space might harbor higher information content. Embeddings from the codon language model have recently demonstrated success in various protein downstream tasks. However, predictive models for residue-level tasks such as phosphorylation sites, arguably the most studied Post-Translational Modification (PTM), and PTM sites prediction in general, have predominantly relied on representations in amino acid space. Results: We introduce a novel approach for predicting phosphorylation sites by utilizing codon-level information through embeddings from the codon adaptation language model (CaLM), trained on protein-coding DNA sequences. Protein sequences are first reverse-translated into reliable coding sequences by mapping UniProt sequences to their corresponding NCBI reference sequences and extracting the exact coding sequences from their GenBank format using a dynamic programming-based global pairwise alignment. The resulting coding sequences are encoded using the CaLM encoder to generate codon-aware embeddings, which are subsequently integrated with amino acid-aware embeddings obtained from a protein language model, through an early fusion strategy. Next, a window-level representation of the site of interest, retaining the full sequence context, is constructed from the fused embeddings. A ConvBiGRU network extracts feature maps that capture spatiotemporal correlations between proximal residues within the window. This is followed by a prediction head based on a Kolmogorov-Arnold network (KAN) using the derivative of gaussian wavelet transform to generate the inference for the site. The overall model, dubbed CaLMPhosKAN, performs better than the existing approaches across multiple datasets. Availability and implementation: CaLMPhosKAN is publicly available at https://github.com/KCLabMTU/CaLMPhosKAN.

Protein post-translational modifications (PTMs) introduce new functionalities and play a critical role in the regulation of protein functions. Characterizing these modifications, especially PTM sites, is essential for unraveling complex biological systems. However, traditional experimental approaches, such as mass spectrometry, are time-consuming and expensive. Machine learning and deep learning techniques offer promising alternatives for predicting PTM sites. In this chapter, we introduce our LMPTMSite (language model-based post-translational modification site predictor) platform, which emphasizes two transformer-based protein language model (pLM) approaches: pLMSNOSite and LMSuccSite, for the prediction of S-nitrosylation sites and succinylation sites in proteins, respectively. We highlight the various methods of using pLM-based sequence encoding, explain the underlying deep learning architectures, and discuss the superior efficacy of these tools compared to other state-of-the-art tools. Subsequently, we present an analysis of runtime and memory usage for pLMSNOSite, with a focus on CPU and RAM usage as the input sequence length is scaled up. Finally, we showcase a case study predicting succinylation sites in proteins active within the tricarboxylic acid (TCA) cycle pathway using LMSuccSite, demonstrating its potential utility and efficiency in real-world biological contexts. The LMPTMSite platform, inclusive of pLMSNOSite and LMSuccSite, is freely available both as a web server (http://kcdukkalab.org/pLMSNOSite/ and http://kcdukkalab.org/LMSuccSite/) and as standalone packages (https://github.com/KCLabMTU/pLMSNOSite and https://github.com/KCLabMTU/LMSuccSite), providing valuable tools for researchers in the field.

Posttranslational modifications (PTMs) play a crucial role in modulating the structure, function, localization, and interactions of proteins, with many PTMs being localized within supersecondary structures, such as helical pairs. These modifications can significantly influence the conformation and stability of these structures. For instance, phosphorylation introduces negative charges that alter electrostatic interactions, while acetylation or methylation of lysine residues affects the stability and interactions of alpha helices or beta strands. Given the pivotal role of supersecondary structures in the overall protein architecture, their modulation by PTMs is essential for protein functionality. This chapter explores the latest advancements in predicting sites for the five PTMs (phosphorylation, acetylation, glycosylation, methylation, and ubiquitination) known to be localized within supersecondary structures. The chapter highlights the recent advances in the prediction of these PTM sites, including the use of global contextualized embeddings from protein language models, integration of structural information, utilization of reliable positive and negative sites, and application of contrastive learning. These methodologies and emerging trends offer a roadmap for novel innovations in addressing PTM prediction challenges, particularly those linked to supersecondary structures.

DNA-binding proteins (DBPs) play critical roles in gene regulation, development, and environmental response across various species, including plants, animals, and microorganisms. While various machine learning and deep learning models have been developed to distinguish DNA-binding proteins (DBPs) from non-DNA-binding proteins (NDBPs), most available tools have focused on human and mouse datasets. As a result, there are limited studies specifically addressing plant-based DNA-binding proteins, which restricts our understanding of their unique roles and functions in plant biology. Developing an efficient framework for improving DBP prediction in plants would enhance our knowledge and enable precise gene expression control, accelerate crop improvement, enhance stress resilience, and optimize metabolic engineering for agricultural advancement. In this work, we developed a tool that uses a protein language model (pLM) pre-trained on millions of sequences. We evaluated several leading models, including ProtT5, Ankh, and ESM-2, and leveraged their high-dimensional, information-rich representations to improve the accuracy of DNA-binding protein prediction in plants significantly. Our final model, pLM-DBPs, a feed-forward neural network classifier utilizing ProtT5-based representations, outperformed existing approaches with a Matthews Correlation Coefficient (MCC) of 83.8% on the independent test set. This represents a 10% improvement over the previous state-of-the-art model for plant-based DBP prediction, highlighting its superior performance compared to the existing approaches.