ESM 3 — The Frontier of Protein Design?

ESM 3 — The Frontier of Protein Design?

TL;DR

ESM-3 offers us sequence, structure and function design all unified in a single model. It appears performant across all three domains, yet remains under an elusive non-commercial license…

ESM3’s generated sequences produce structures with realistic foldability (TM-score 0.52 ± 0.10) that are reliable on a lower level (pLDDT 0.84), as well as coming from a diverse range of sequences that share little similarity (mean pairwise sequence identity of 0.155). While not directly compared in the paper, it is worth noting that ESM3s TM-scores are much lower than some of its predecessors, such as the aforementioned ProGen2 which has a mean TM-Score of 0.72. It is worth noting that this may be from ESM3’s tendency to generate new proteins that are ‘distant’ from known ones, which would adversely affect its TM-Scores.

Functionally, ESM3s generated proteins maintain coordination and active sites with high designability (pTM > 0.8, scTM 0.96 ± 0.04) and shows a strong ability to follow complex prompts including sequence, structure coordinates, secondary structure (SS8), solvent-accessible surface area (SASA), and function keywords. The table below shows how well ESM-3 predicts each modality, given the others as inputs.

Modality

Some Critiques

Results, Schmeults, amiright? ESM3 may not perform as well across all benchmarks as models specifically tuned for that task, and it’s hard to expect it to with everything else it does. There’s also the question of size, where out-performance of ESMFold begins at the price of 7B parameters. It’s worth noting that many models tuned to a particular domain, such as ESM-2, are still really really good at that particular task, and they tend to do it with a hell of a lot less parameters. Unless there’s a specific need for in-built structure generation and function prediction as well, it makes one wonder why 98B parameters is justifiable over 650M.

And there is the question of access… At the moment EvolutionaryScale are keeping ESM-3 tight to their chests, meaning it’s strictly available under a non-commercial license, meaning hardly anyone can use it. With our hands all over ESM-3 we could get a better understanding of its capabilities and limitations, though for now we’ll just have to go with what they’ve given us — it appears to be a completely performant model over the sequence, structure and functional domains, without a particular expertise in any.

Let’s hope that in future our benevolent overlords at EvolutionaryScale change their minds and open their models to the masses. At the end of the day, what have they got to lose?

Conclusion

Only time will tell if the proteins ESM3 makes are a little too distant from those known to us to be practically useful. Has mother nature evolved this way for a reason, or is she simply missing the point? Until ESM3 unlocks the secrets to reverse ageing and cure every disease on Earth, we‘ll never know. To ensure you’re as up to date as can be with all that is Machine Learning, make sure to stay tuned, and if you’re interested in any of our work you can find it here.

For more information, or if you think you could’ve done a better job and want to tell me why, you can find me here: [email protected].

Good luck out there.

ESM 3 — The Frontier of Protein Design? was originally published in ML6team on Medium, where people are continuing the conversation by highlighting and responding to this story.