Ask HN: 如何设置本地 LLM 备份？
3 分•作者: andy99•10 个月前
为了以防未来访问权限发生变化，我想备份最先进的 LLM 权重，并准备一台可以运行它们的机器。OpenAI 发布权重是一个很好的时机来实现这个目标。我的问题是，你会购买什么样的硬件配置，它应该价格合理（比如 5000 美元以下，最好远低于这个价格），并且能够很好地运行本地查询模型。如果重要的话，它也应该适合存档，比如直到出现重大技术进步，导致今天的 LLM 过时，或者直到因为无法再获得优秀的、开放的权重时才需要用到。
查看原文
As a backup in case future access changes, I'd like to have sota LLM weights and a machine that can run queries with them in reserve. OpenAI releasing weights is as good a time as any to actually do it. My question is what hardware setup would you buy, that is reasonable accessible (say under 5k, ideally well under) and can do a good job running the models for local queries. And, if it matters, could be suitable for archiving, say until either there is a substantial advance rendering today's LLMs obsolete or until needed because good, open weights aren't available anymore.