HN 提问：谁在使用 FPGA 进行机器学习推理？
3 分•作者: softwarewright•23 天前
随着内存价格上涨，我想知道 FPGA 是否可以用于卸载推理处理，而无需将权重保存在内存中？可用的内存将用于激活、KV 缓存、上下文，而不是静态权重。权重可以从磁盘流式传输。这种方法不是为了提高每秒的 token 数，而是为了以更低的成本提高吞吐量。可能更好的答案/kHh？我已经开始研究这个问题，但想知道其他人是否考虑过/尝试过？
查看原文
With RAM price inflation, I wonder if FPGAs can be used to offload inference processing without keeping weights in RAM? The available RAM would be for activations, KV Cache, context but not static weights. Weights could be streamed from disk. This approach is not for tokens/second but throughput at a lower cost. Possibly better answers/kHh? I've started researching this, but wonder if others have considered/tried this?