Ask HN：这里有谁在生产环境中使用本地 LLM？
3 分•作者: Haeuserschlucht•6 个月前
我之所以这么问，是因为似乎真的没人这么做。是的，偶尔会有一些项目，但最终大家都会转向云端大语言模型。一切都在云端。人们为在不知名的地方使用 GPU 付费。但实际上，没有人长期使用本地大语言模型。他们会说：“嗯，这太棒了。本地大语言模型可以在小型设备上运行，甚至可以在你的手机上运行。”
我得说，对我来说，有一个例外，那就是 Whisper。我确实经常使用 Whisper。但我就是不用本地大语言模型。与云端 GPU 相比，它们真的、真的差劲。
我不知道为什么会这样，因为在我看来，创建一个语音转文本模型比创建一个生成文本的模型要难得多。
但似乎他们真的无法消除这种差距，让它在消费级电脑上运行。所以，抛开隐私不谈，我也回到了云端大语言模型。
查看原文
I'm asking because it seems that nobody really does that. Yes, there are some projects here and there, but ultimately everybody just jumps over to cloud LLMs. Everything is cloud. People pay for GPU usage somewhere in the middle of nowhere. But nobody really uses local LLMs long term. They say, "Well, it's so great. Local LLMs work on small devices they even work on your mobile phone."<p>I have to say there's one exception for me and that's Whisper. I actually do use Whisper a lot. But I just don't use local LLMs. They're just really, really bad compared to cloud GPUs.<p>And I don't know why, because for me it seems that having a speech-to-text model is much more challenging to create than just a model that creates text.<p>But it seems that they really cannot remove the differences and have it run on consumer computers. And so I also go back to cloud LLMs, all privacy aside.