Recently, the Enzyme Products UK AI R&D Center and Professor Liu Jian’s research team at the University of Birmingham jointly published an article entitled “ProCeSa: Contrast-Enhanced Structure-Aware Network for Thermostability Prediction with Protein Language Models” in the internationally renowned academic journal Journal of Chemical Information and Modeling (impact factor 5.7) . This article showcases the Enzyme Products research team’s strength and latest research results in the field of AI-assisted synthetic biology engineering.
Original link: https://doi.org/10.1021/acs.jcim.4c01752
1. Research Project Overview
Predicting protein thermostability is crucial in fields such as biotechnology, enzyme engineering, and drug design. While traditional experimental methods are reliable, they are often time-consuming, labor-intensive, and costly. This research project proposes a novel protein thermostability prediction model, ProCeSa (Protein Contrast-enhanced Structure-aware), which aims to accurately predict protein thermostability directly from its sequence.
ProCeSa innovatively combines protein language models (PLMs) with contrastive learning to capture sequence and structural information without requiring the display of the protein’s three-dimensional structure. The model first generates initial sequence representations and contact maps using pre-trained PLMs, and then learns structural features through a graph convolutional network (GCN). The structural and sequence representations are then used as positive and anchor representations, respectively, while negative representations are constructed through strategic amino acid sampling. This approach helps the model learn the intrinsic relationships between sequence and structural features that influence protein stability.
Compared to existing methods, ProCeSa avoids reliance on detailed structural information and overcomes the potential inaccuracies of structure prediction-based methods. This research not only advances computational biology but also provides new insights for protein engineering and design. Its practical applications are significant, potentially leading to breakthroughs in areas such as industrial enzyme design and screening, and the development of therapeutic proteins.
2. ProCeSa Algorithm Overview
The ProCeSa algorithm first uses pre-trained protein language models (PLMs) to encode features of the input sequence. Then, a multilayer perceptron (MLP) maps the high-dimensional features to a learning space to obtain the sequence’s implicit representation. For structural information modeling, a spatial feature extractor based on the interactions between amino acid residues is constructed using a graph convolutional neural network (GCN), and a symmetrically normalized contact graph (extracted from the protein language model) is introduced to effectively capture the local and global structural relationships between residues.
ProCeSa structural diagram
The core innovation of the algorithm lies in its triplet-based contrastive learning strategy, which uses carefully constructed positive and negative sample pairs for representation learning. Specifically, a fixed number of amino acid residues are sampled from sequence and structure representations. Residue pairs with the same category label are used as positive samples, while residues with different category labels are selected to construct a negative sample set. This strategy not only enhances the model’s understanding of sequence-structure relationships but also improves its ability to distinguish between different thermal stability characteristics.
In the feature aggregation stage, ProCeSa employs a self-attention pooling mechanism, adaptively integrating amino acid features of proteins of different lengths through learnable attention weights. Finally, by jointly optimizing the contrastive and predictive losses, the algorithm achieves sequence-to-thermal stability prediction while maintaining the model’s generalization ability. This multi-level feature learning and optimization strategy enables ProCeSa to accurately predict protein thermal stability without relying on complex structural information.
3. Experimental Results
This study compared ProCeSa with five other algorithms on the HotProtein dataset.
(1) Classification task: On various subsets of the HotProtein dataset, the ProCeSa model outperforms other baseline models (Table 1). By combining the state-of-the-art ESM-C model with contrastive learning, the accuracy of ProCeSa is further improved.
The ROC curves on the HP-S2C5 subset (Figure 2) show that ProCeSa, using contrastive learning, outperforms other models in accuracy. Analysis of individual ROC curves for the five protein categories in HP-S2C5 (Figure 3) reveals different trends; ProCeSa-ESM-C exhibits balanced accuracy across all categories, demonstrating its robustness. Most models achieve the highest AUC in the low-temperature category, while ProCeSa-ESM-C performs best in the high-temperature category, likely because ESM-C better captures features associated with high-temperature proteins.
Detailed data is shown in the table below:
(2) Regression Task: On various subsets of the HotProtein dataset, ProCeSa using ESM-C and contrastive learning achieved the highest prediction accuracy. These results highlight the effectiveness of ESM-C in improving representation quality and the ability of contrastive learning to improve the modeling of sequence and structural features.
Detailed data are shown in the table below:
(3) Generalization ability: On the DeepStabP dataset, the performance of the ProCeSa-ESM-C model is comparable to that of the DeepStabP model based on ProtT5-XL PLM, but the number of parameters is significantly reduced (600M vs 3B), which reflects the generality of the model.
4. Summary
ProCeSa is a novel protein thermostability prediction model that innovatively combines protein language models with contrastive learning to achieve high-accuracy predictions without relying on detailed 3D structural data. The model employs a sequence-structure dual-stream feature extraction architecture, extracting sequence features through pre-trained PLMs and MLPs, while utilizing GCN to construct an amino acid residue relationship graph to obtain structural information. The core of the model lies in its innovatively designed triplet contrastive learning framework, which enhances feature representation capabilities by constructing sequence-structure positive sample pairs and strategically sampling negative samples. Finally, a self-attention pooling mechanism is used to achieve a unified representation of proteins of different lengths, and accurate thermostability predictions are achieved through end-to-end training. Experiments show that ProCeSa outperforms other methods on the HotProtein dataset, making it a powerful computational tool for protein engineering and design.
5. Regarding finished enzyme products
Enzyme Products is committed to becoming a global leader in green active ingredients, employing next-generation synthetic biology technology to focus on the green production of bioactive ingredients such as peptides, proteins, sugars, nucleic acids, and lipids. Leveraging over 20 years of research and industry experience accumulated by its founding team, the company has established a leading multidisciplinary technology platform in China. It has achieved large-scale production of dozens of green active ingredients, including Smegglutinin, S -Pros-Xylane, Ergothioneine, Ectoin, Collagen Tripeptide, Human Milk Oligosaccharides, Ceramides, NAD+, NMN, Copper Peptide, Carnosine, and Glutathione. It has cumulatively provided high-quality green active ingredients to over 2,000 well-known companies in the fields of functional skincare, biomedicine, nutrition and health, and animal nutrition .
The company ranks among the top international companies in terms of R&D investment, scale, and output value. It has undertaken a number of key provincial and municipal R&D projects, applied for nearly 200 invention patents, and received more than ten national and local titles, including National Specialized, Refined, and Innovative Small Giant Enterprise, National High-tech Enterprise, Guangdong Provincial Postdoctoral Innovation Practice Base, Shenzhen Peacock Team, Shenzhen Potential Unicorn Enterprise, Zhuhai Innovation and Entrepreneurship Team, and Gansu Provincial Specialized, Refined, and Innovative Small and Medium-sized Enterprise. It has also been recognized as one of the Top 20 High-Growth Enterprises in Shenzhen in 2024, the Harvard Business Review High-Energy Innovation Team Award in 2023, and one of the Top 100 High-Growth Enterprises in Shenzhen in 2022.
References:
– Code: https://github.com/notabigfish/procesa
– Original article: Zhou, F.; Zhang, S.; Zhang, H.; Liu, J. ProCeSa: Contrast-Enhanced Structure-Aware Network for Thermostability Prediction with Protein Language Models. J. Chem. Inf. Model. 2025, DOI: https://doi.org/10.1021/acs.jcim.4c01752