We tried to beat Pfam for BGC retrieval. Pfam won.

The full article is on Medium. This is the short version.

Protein language models encode things a hand-designed feature list does not, so it seemed reasonable to ask whether they could improve retrieval of related biosynthetic gene clusters. Given a few known BGCs from a reference group, can we rank the remaining candidates and find the related ones?

I built the pipeline end to end: Streptomyces griseus genomes from NCBI, Prodigal ORF calling, DeepBGC cluster detection, BLASTp against MIBiG for silver labels, and genome deduplication. That gives an atlas of 6,953 BGCs from 182 deduplicated genome accessions, of which 5,325 carry a silver assignment to a MIBiG reference group. The learned model — Pfam-augmented BGC-SetNet — aggregates frozen 1,280-dimensional ESM-2 gene embeddings with a Set Transformer and also sees a BGC-level inventory of Pfam domains. It was compared against raw mean-pooled ESM-2, plain Pfam Jaccard, an ensemble of the two learned scores, and a differentiable weighted-Pfam Jaccard.

The evaluation came first

Reference groups were split into training, validation and test before any training, pair construction or query generation. Checkpoints, model choice and ensemble weights were all selected on validation alone. Five optimisation seeds, and inference paired at the level of the biological reference group rather than the individual query, because the group is the unit that is actually independent.

What happened

• Plain Pfam Jaccard: Recall@50 0.8788
• Pfam-augmented BGC-SetNet: 0.8472
• Raw ESM + BGC-SetNet ensemble: 0.8769, with no statistically supported improvement over the baseline
• Learned weighted-Pfam Jaccard: 0.8789

That last number is higher than the baseline by 0.0001. I treat the two as numerically indistinguishable rather than as a win.

Only 16 held-out groups had more than the four references the retrieval protocol needs, so the effective evaluation set is small, and the labels are silver rather than curated pathway-level ground truth. Both limitations are stated in the paper instead of being replaced with a more favourable subgroup.

What it means

Domain identity already explains most of the signal this benchmark measures. That does not make sequence representations useless for BGC work — it means this particular task leaves them nothing to add. Sequence-derived features have a real opportunity where domain annotations are incomplete, where divergent enzymes keep a function without keeping obvious domain similarity, or where the endpoint is chemistry rather than recovering proxy groups that were built out of known reference information in the first place. Those experiments need labels that can carry the claim.

Preprint: doi.org/10.64898/2026.08.21.746127