HN 提问：你们在生产环境中使用什么来进行 LLM 推理？
2 分•作者: chrislim•8 天前
我最初在 GPT-3 时代加入了 OpenAI，之后辗转于各大实验室（期间短暂地使用过 Workers AI）。
最终我选择了 Gemini 2.5 Flash Lite 作为我的主力模型，将其用于处理结构化数据/向量并作为聊天机器人。它价格便宜，性能足够好，最重要的是，它**速度很快**——但现在它即将被淘汰，我不知道去哪里才能找到类似的性能。
Groq 看起来很有前景，但开发者访问权限
查看原文
I started with OpenAI back in the GPT-3 days, then bounced between the major labs (with a brief interlude with Workers AI).<p>I eventually settled on Gemini 2.5 Flash Lite as my workhorse, using it against structured data/vectors as a chatbot. It was cheap, good enough, and most importantly, <i>fast</i> - but it's now being sunset and I'm not sure where to go to find similar performance.<p>Groq seems promising but developer access hasn't been available for months. I'd love to use open source, but I never found anything comparable (cost/performance/speed).<p>Would love to hear your suggestions.