In December 2023, Professor Jian Liu’s research team from the UK AI R&D Center for Enzyme Products and the University of Birmingham published a high-quality long article entitled “HybridGCN for protein solubility prediction with adaptive weighting of multiple features” in the internationally renowned academic journal Journal of Cheminformatics (impact factor 8.6), showcasing the team’s latest research results in the field of AI-assisted biochemical engineering.
The following is an analysis of the paper:
Research Project Overview
Protein solubility is a crucial biophysical property, essential for assessing protein effectiveness in biological and chemical engineering, and a major factor influencing drug research and yield. Poor protein solubility affects protein production, and the primary structure of a protein, particularly its amino acid sequence, is a primary determinant of protein solubility. Numerous studies have demonstrated a strong correlation between protein solubility and sequence-based characteristics; therefore, exploring predictive techniques for estimating solubility using sequence information is crucial.
To address these issues, the research team proposed and implemented a protein solubility prediction method based on multi-feature adaptive weighted fusion GCN, called HybridGCN. HybridGCN is a unified framework based on graph convolutional neural networks that leverages the advantages of fully integrating artificial and deep features to predict protein solubility.
Most existing models are designed for binary classification tasks, categorizing datasets into soluble and insoluble classes, ignoring the continuity of solubility. In protein engineering, continuous solubility values are more important than binary classification because they provide more guidance for downstream tasks. For example, in large protein datasets, the optimal protein sequence can be selected based on continuous solubility values, a task that binary values cannot accomplish. Furthermore, traditional machine learning models lag behind deep learning models in performance due to the limited generalization ability of handcrafted features. Deep learning models have achieved state-of-the-art performance on various protein engineering tasks, including structure prediction, protein design, protein conjugate design, stability prediction, and solubility prediction. Therefore, understanding how to combine different biophysical features with deep learning features in a more flexible way into a single model is a viable solution.
HybridGCN Prediction Framework
First, a zero-shot learning ESM-1v1 model is used to extract deep features from protein data. ESM-1v1 features capture broad and relevant information related to protein function and structure, thus aiding in protein solubility prediction. These deep features are combined with five handcrafted features as input to the HybridGCN framework. After entering the framework, the mixed data first passes through an Adaptive Feature Reweighting (AFR) module, which integrates domain knowledge from handcrafted features with deep cues extracted from the deep learning model. This is followed by a GCN module, an attention module, and finally a fully connected layer and sigmoid mapping to the predicted solubility. The method is applied to the eSOL and S. cerevisiae datasets, which contain real-world solubility values. Results show that this method focuses on the interaction between deep features and classic handcrafted features, and observes that certain classic features, particularly evolutionary features, complement deep features, significantly improving the accuracy of protein solubility prediction and achieving state-of-the-art prediction results.
Figure 1: HybridGCN Framework DiagramTo verify the contribution of the proposed module to the overall model performance, the research team conducted ablation experiments, specifically removing the AFR module and ESM-1v feature separately from HybridGCN, resulting in two distinct models: the ESM+GCN model and the AFR+GCN model. Compared to using GCN alone, adding the ESM-1v feature to GCN significantly improved R² (0.493) and recall (0.713). This significant performance improvement was primarily due to the ESM-1v module, which, as a powerful zero-shot learning function, absorbed information related to protein structure and function. This further reinforces the concept that protein solubility is closely related to protein structure and function. The AFR module also improved the performance of GCN on all evaluation metrics. These findings suggest that not all individual features are equally important; the AFR module learns a superior composite feature that effectively highlights the most informative features related to protein solubility .
Figure 2: Performance of HybridGCN and GCN at different training stages
Figure 2 illustrates the superiority of HybridGCN over GCN across different training phases and various evaluation metrics. Overall, HybridGCN performs best in all four settings for both regression and classification metrics. The observed performance improvements can be attributed to the contributions of ESM-1v features and the AFR module. These results highlight the importance of well-designed feature engineering strategies in protein solubility prediction tasks.
Performance Comparison
On the eSOL dataset, this method is compared with several current protein solubility prediction methods, and the results are shown in Table 1. Both GraphSoLnsemble and GraphSoLSingle use Graph Convolutional Networks (GCNs) as the network architecture and utilize various traditional features as inputs, including physicochemical features (AAPHY7), evolutionary features (PSSM and HMM), structural features (SPIDER3), and Blosum62 features. The authors of GraphSoL conducted extensive ablation experiments, showing that some features are more important for the solubility prediction task, and evolutionary features are more critical than structural features. These observations inspired this method in two ways: (1) introducing the AFR module, which can adjust the importance of features according to their correlation with solubility prediction; (2) requiring more powerful input features, such as ESM-1v features, which capture rich information related to protein structure and function. By combining the AFR module and ESM-1v features, this HybridGCN outperforms all previous methods on the eSOL dataset.
SeqVec and TAPE are transfer learning frameworks for solubility prediction that utilize deep features or embeddings from pre-trained deep networks as input to the solubility prediction task. However, their performance is inferior to HybridGCN due to the lack of traditional features (such as evolutionary features) that can complement deep features. DeepSoL integrates additional biological and structural features to improve deep features, but its performance is limited by its use of a basic convolutional neural network as the network architecture. Although it incorporates features from multiple sources, it relies solely on simple connection operations to combine these features, which may not effectively select and enhance the most informative features for solubility prediction. In contrast, HybridGCN is a GCN-based network architecture that directly processes graph structures and utilizes structural information from proteins. Due to the introduction of the AFR module and ESM-1v features, HybridGCN consistently outperforms all other models on all metrics. HybridGCN is compared with other high-performance methods on the S. cerevisiae dataset, and the results are shown in Table 2. The results show that OurEnsemble achieved the best R² result (0.390) among the comparison methods, indicating that HybridGCN outperforms other methods in modeling feature relationships. Notably, OurSingle (R²=0.378) outperformed GraphSolEnsemble (R²=0.372) on the S. cerevisiae dataset. Although the performance gain was small, this single model was more efficient than the ensemble model during inference.
Summarize
The research team proposed a novel graph neural network, HybridGCN, which combines deep features with classic solubility-related features to improve the accuracy of protein solubility prediction. The model considers the structural and biological characteristics of protein sequences, as well as deep features extracted from high-capacity deep learning models to enhance prediction performance. Analysis identifies interactions between deep features and classic features, where certain classic features complement deep features in the solubility prediction task. To further improve the prediction task, ESM-1v features, or zero-shot learned features, are introduced to obtain comprehensive and relevant information about protein function and structure. Furthermore, an adaptive feature weighting (AFR) module is proposed to explore feature interactions and enhance the most informative features in solubility prediction. Ablation experiments and comparisons demonstrate the effectiveness of ESM-1v features and the AFR module. HybridGCN achieves state-of-the-art performance on publicly available datasets, predicting continuous solubility values.
Regarding enzyme products
Enzyme Products is committed to becoming a global leader in green active ingredients, employing next-generation synthetic biology technology to focus on the green production of active ingredients such as peptides, proteins, sugars, and nucleic acids. Leveraging over 20 years of research and industry experience from its founding team, the company has established a leading multidisciplinary technology platform in China. It has now achieved mass production of dozens of green active ingredients, including Smegglutinin, HMOs, copper peptides, NMN, S-type serotonin, NAD+, and ergothioneine, providing high-quality green active ingredients to nearly a thousand well-known companies in the fields of functional skincare, biomedicine, and nutritional health. After nearly six years of rapid development, the company has achieved a leading position among domestic enterprises in terms of R&D investment, scale, and output. It has applied for nearly 100 invention patents and has won more than ten national and local titles, including National Specialized and Innovative Small Giant, National High-tech Enterprise, Shenzhen Specialized and Innovative Small and Medium Enterprise, Shenzhen Potential Unicorn Enterprise, Shenzhen Postdoctoral Innovation Practice Base, Shenzhen Top 30 Medical and Health Innovation Team, Zhuhai Innovation and Entrepreneurship Team, and Gansu Specialized and Innovative Small and Medium Enterprise. In 2022, it was selected as one of Shenzhen’s Top 100 High-Growth Enterprises (ranked first among unlisted biotech companies), and in 2023, it was selected as a Harvard Business Review High-Energy Innovation Team.
References
The dataset used in this paper is publicly available data: https://github.com/jcchan23/GraphSol/tree/master/Data
The code is publicly available on GitHub: https://github.com/IanDragon
Please refer to the original text:
Chen, L., Wu, R., Zhou, F. et al. HybridGCN for protein solubility prediction with adaptive weighting of multiple features. J Cheminform 15, 118 (2023). https://doi.org/10.1186/s13321-023-00788-8