Towards General Protein Structure Representation Learning With a Protein Size Prompt

Authors:
Publisher website
Google Scholar
Abstract
Deep learning-based protein function analysis is rapidly advancing, but most methods rely heavily on labeled data, which is typically scarce due to the need for expensive wet lab experiments. Self-supervised learning (SSL) is a viable solution because it can improve the model performance on downstream tasks by pre-training on large-scale unlabeled data. However, existing SSL methods overlook the disparities in data distributions between pre-training and downstream tasks, which we show has a large influence on the effectiveness of SSL. To tackle this problem, we propose a new protein representation learning framework with a protein size prompt (ProSP). ProSP allows the pre-trained model to learn size-related information during pre-training and adapt to downstream tasks with different protein size distributions. Results of 48 experiments across eight downstream tasks, three data splits, and two data representations show that ProSP successfully improves the performance in all experiments with minimal modifications to the original model.
Funding Information
  • Science and Technology Innovation Plan Of Shanghai Science and Technology Commission (24J22800800)
  • Fudan University Science Intelligence Special Fund (FD-AI4S04183)

This publication has 44 references indexed in Scilit: