我对“人类灭绝事件炒作列车”的反驳
1 分•作者: hmichaelson24•16 天前
“两家公司都没有负责任地行事。它们正朝着自我改进的超级智能狂奔，拿我们的生命冒险。”
你可能昨天看到了来自 Anthropic 研究员 Jacob Coxon 的类似帖子，他因担心人工智能可能导致灭绝事件而辞职抗议。这些是我看到的一些相关引述，我想分享一下我对它们的看法：
1. “职位越高的人，越害怕。”
这更多地反映了硅谷的文化和激励机制，而不是其他任何事情。人工智能的神话在硅谷已经自成一派，而经济和监管激励措施都在助长其发展。
2. “我们确实真诚地相信人工智能可能会杀死全人类！我个人认为在未来十年内发生的可能性超过10%。”
有什么证据吗？有没有任何隐含的数学依据来支持这10%？我没有看到任何，只有对超级人工智能的模糊幻想。
3. “Coxon 离开了 Anthropic，尽管公司即将上市，但他放弃了巨额财富，因为他坚信自己的信念。”
首先，坚信不代表正确。而且，即使以 Anthropic 目前的估值来看，上市前的期权也并非真正的牺牲，它们仍然价值非凡。我不相信这种殉道者的叙事。
4. “自我改进的人工智能即将来临，并遵循指数级改进曲线。”
在我看来，这是隐藏在大量叙事包装下的真相。是的，人工智能无疑已经取得了长足的进步。然而，在每一个诸如纳维-斯托克斯方程解法之类的头条新闻背后，都隐藏着一个比标题暗示的更实际、更细致、更具争议的故事。
5. “我们正接近奇点。”
……指的是 OpenAI 和 Anthropic 的首次公开募股（IPO）。
这些极端叙事真正令人遗憾的部分在于，我认为它们实际上适得其反。它们模糊了我们可以解决的具体、实际的问题，并将细致的立场简化为两个对立面。例如，生物武器、自主武器系统、关键基础设施中的人工智能以及大规模网络攻击是当前真正需要严肃技术和政策关注的风险。
当一切都可能成为末日时，就没有什么能得到解决。
查看原文
"Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."<p>You may have seen a post like this yesterday from an Anthropic researcher named Jacob Coxon who quit the company in protest over fears that AI may be an extinction level event. These are a few of the quotes I saw surrounding it, and I want to share my own takes on them:<p>1. "The more senior the people, the more scared they are"<p>This says more about Silicon Valley culture and incentives than anything else. The mythology around AI has taken on a life of its own in the Valley, and the financial and regulatory incentives support it growing.<p>2. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."<p>Any evidence? Any sort of even implied math behind the 10%? I haven't seen any, only vague illusions to ASI.<p>3. "Coxon left Anthropic, leaving so much money on the table in spite of the upcoming IPO because he's earnest in his beliefs"<p>First off, earnest doesn't mean correct. And pre-IPO options at Anthropic's current valuation even now aren't exactly a sacrifice, they're still worth an extraordinary amount. I don't believe the martyrdom framing.<p>4. "Self-improving AI is coming and following the exponential improvement curve"<p>In my opinion, this is a kernel of truth hidden under a mountain of narrative spin. Yes, AI has undoubtedly gotten much better. However, behind every headline such as the Navier-Stokes solution, there is a much more grounded, nuanced, and controversial story than the headline might suggest.<p>5. "We are approaching the singularity"<p>...of OpenAI and Anthropic's IPOs<p>The really unfortunate part of these extreme narratives is I believe they are actually counterproductive. It obscures the specific, tangible problems that we can solve in extreme language collapsing nuanced positions into two sides. For example, bioweapons, autonomous weapons systems, AI in critical infrastructure, and large-scale cyberattacks are real risks that deserve serious technical and policy attention right now.<p>When everything is a potential apocalypse, nothing gets fixed.