HELM-BERT model achieves superior cyclic peptide membrane permeability prediction, outperforming SMILES-based encoders

Developing effective computational models for chemically modified and macrocyclic peptides is challenging due to their complex structures. Traditional molecular representation models, such as atom-level strings (SMILES) or protein sequence models, struggle to unify chemical modifications and covalent topology. SMILES obscures macrocyclic connectivity, while protein sequence models cannot encode noncanonical residues or explicit cross-links. This gap hinders the in silico prediction of critical properties like membrane permeability and peptide-protein interactions, slowing down the discovery and optimization of these increasingly important therapeutic agents.

Researchers pretrained an encoder-only transformer model, named HELM-BERT, directly on Hierarchical Editing Language for Macromolecules (HELM) notation, which explicitly specifies monomer identity and connectivity. The model was evaluated on its ability to predict cyclic peptide membrane permeability using both random and Murcko scaffold splits. Performance was compared against external pretrained SMILES-based encoders and an architecture-matched SMILES control. Additionally, HELM-BERT's capability for peptide-protein interaction prediction was assessed using complementary Propedia and ChEMBL benchmarks.

HELM-BERT achieved the best mean performance in cyclic peptide membrane permeability prediction, yielding a random split R2 = 0.668. This performance exceeded that of external pretrained SMILES-based encoders. The model also retained its superior mean performance under a Murcko scaffold split, indicating robustness to structural diversity. An architecture-matched SMILES control narrowed the performance gap when subjected to full finetuning, however, HELM-BERT maintained clearer advantages in frozen-representation settings.

HELM-BERT successfully preserved HELM-specified macrocyclic topology in a linearly accessible form, supporting competitive peptide-protein interaction prediction across both Propedia and ChEMBL benchmarks. These findings suggest that pretraining on notation explicitly encoding structural constraints offers a transferable strategy for biomolecular modalities that bridge the gap between small-molecule chemistry and protein sequences.