HackerNews中文版

华尔街日报 Anthropic 公司呼吁顶尖人工智能实验室考虑放缓发展步伐，认为人工智能系统正以惊人的速度进步，可能很快就能在无需人类干预的情况下自我改进，从而带来重大的社会风险。该公司周四在一篇博文中表示，能够减缓全球人工智能发展“很可能是一件好事”。该博文披露了内部数据，记录了其最先进模型改进的速度。这篇由其内部研究所所长和政策主管撰写的博文指出，模型的进步似乎正朝着“递归自我改进”的方向发展，即人工智能系统可以在没有人类干预的情况下自行改进。一些人工智能内部人士认为，这一门槛可能预示着危险和巨大的社会动荡。该博文由 Marina Favaro 和 Jack Clark 撰写，表示“我们相信，让世界有机会放缓或暂时暂停前沿人工智能的发展，以便社会结构和对齐研究能够跟上技术进步的步伐，这将对世界有利。” 文章提议就如何可能放缓发展达成全球协议，并建立一种机制来验证竞争对手是否遵守该协议。该博文警告说，递归自我改进尚未发生，也并非不可避免，“但可能比大多数机构准备的要来得更早。” Anthropic 公司最近完成了一轮融资，公司估值接近 1 万亿美元，并已提交保密文件，开始公开上市的流程。该公司最近已成为与 ChatGPT 制造商 OpenAI 在人工智能霸权激烈竞争中的领跑者，OpenAI 也预计将很快提交首次公开募股的文件。 Anthropic 的运行率（初创公司常用的预测短期销售额年度收入的指标）有望在本月底达到 500 亿美元的年化收入，高于 2025 年底的 90 亿美元。

查看原文

WSJ Anthropic is calling for top artificial intelligence labs to weigh slowing the pace of development, suggesting that AI systems are advancing so rapidly that they may soon be able to improve themselves without human intervention in ways that could pose significant societal risks.The ability to slow global AI development would “likely be a good thing,” the company said Thursday in a blog post that disclosed internal data documenting how quickly its most advanced models are improving.The post, written by the head of its internal research institute and head of policy, noted that model advances appear to be on a path toward “recursive self-improvement,” when AI systems can improve on their own without human intervention. Some AI insiders have seen that threshold as a potential marker of danger and enormous societal upheaval.“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” the post, written by Marina Favaro and Jack Clark, says. It proposes a global agreement on how to potentially slow development and a mechanism for verifying that competitors are respecting it.The post cautions that recursive self-improvement hasn’t yet happened and isn’t inevitable, “but could come sooner than most institutions are prepared for.”Anthropic recently concluded a fundraising round that valued the company at almost $1 trillion and filed confidential paperwork to begin the process of publicly listing its shares. The company has recently emerged as the front-runner in a ferocious competition for AI supremacy with ChatGPT-maker OpenAI, which is also expected to file paperwork for an initial public offering soon.Anthropic’s run-rate, a figure commonly used by startups that forecasts annual revenue based on short-term sales, is on track to reach $50 billion in annualized revenue by the end of this month, up from $9 billion at the end of 2025.

Anthropic 呼吁全球暂停人工智能开发，警告“自我改进”风险