HN 提问：科技行业是否需要确定性的机器学习训练？
4 分•作者: Swapnopam•2 个月前
您好，我想了解一下，科技行业是否需要确定性的机器学习训练，还是可以接受非确定性？
也就是说，我知道在大型语言模型（LLMs）的情况下，训练过程中微小的浮点数差异通常是可以容忍的，因为模型最终会收敛到相似的质量，无论这些差异如何。但在更具体的领域，例如医疗器械认证、自动驾驶汽车安全验证或金融模型审计，我觉得在相同的硬件上，两次训练运行是否能产生位相同的模型至关重要，更不用说在异构硬件上。我读到过在某些情况下，例如高频交易（HFT）或量化交易，人们会使用整数来确保确定性的操作，不知道在这里是否也是同样的情况？
查看原文
Hey, I was wondering whether the tech industries require deterministic ML training or is non-determinism acceptable?
As in, I know in case of LLMs, small float differences while training are generally tolerated as the model converges to similar quality regardless. But in more specific domains like medical device certification or autonomous vehicle safety validation or financial model auditing, I feel like whether two training runs producing bit identical models would be critical whether two training runs producing bit-identical models is critical, on the same hardware, let alone across heterogeneous hardware. I have read how in certain cases like HFT or Quant, people would just use integers to make deterministic actions, wondering if its the same case here too?