Ask HN: 如何在页面刷新后保留 LLM 的 token 流？
1 分•作者: spruce_tips•7 个月前
和你们许多人一样，我正在使用服务器发送事件（SSE）从大型语言模型（LLM）那里逐个token地流式传输响应。<p>通过页面刷新来维持SSE连接的最佳方法是什么？<p>我没有看到很多关于这方面的文档或示例。在我使用过的大多数支持LLM的应用程序中，如果token正在流式传输，并且页面刷新/更改，流就会中断。<p>我有一个想法是将流式传输的token写入某种队列或Kafka主题，然后将我的UI连接到队列，并从那里流式传输token。但这似乎需要大量工作。<p>大多数人都是怎么做的呢？
查看原文
Like many of you, I'm streaming responses token by token from LLMs using server sent events (SSEs).<p>What's the best way to maintain the SSE connection through a page refresh?<p>I haven't seen a lot of documentation or examples covering this. In most LLM enabled apps I've used, if tokens are currently streaming and the page refreshes/changes, the stream gets interrupted.<p>One idea I had was writing the streamed tokens into some sort of queue or kafka topic, then connecting my UI to the queue and streaming tokens from there instead. But that seems like a lot of work.<p>How are most folks doing this?