用 165 美元训练 25 个物种的 mRNA 语言模型
24 分•作者: maziyar•4 个月前
我们构建了一个端到端的蛋白质AI流程，涵盖了结构预测、序列设计和密码子优化。在比较了多种用于密码子级别语言建模的Transformer架构后，CodonRoBERTa-large-v2脱颖而出，其困惑度为4.10，Spearman CAI相关系数为0.40，显著优于ModernBERT。随后，我们将规模扩大到25个物种，在55个GPU小时内训练了4个生产模型，并构建了一个物种条件系统，这是其他开源项目无法提供的。完整的成果、架构决策和可运行代码如下。
查看原文
We built an end-to-end protein AI pipeline covering structure prediction, sequence design, and codon optimization. After comparing multiple transformer architectures for codon-level language modeling, CodonRoBERTa-large-v2 emerged as the clear winner with a perplexity of 4.10 and a Spearman CAI correlation of 0.40, significantly outperforming ModernBERT. We then scaled to 25 species, trained 4 production models in 55 GPU-hours, and built a species-conditioned system that no other open-source project offers. Complete results, architectural decisions, and runnable code below.