HackerNews中文版

大家好，我是潜水员，用一个匿名账号来分享我的想法，一个默默无闻的天才……我认为这是一个分享想法的好地方…… 首先，我先提出以下观点并假设其为真：人工智能（AI）为了得出有依据的结论（即决策），会阅读、观察和聆听比任何人类都多得多的信息，然而人类却有能力做出相同甚至更好的决策。我提出以下理论：当人工智能确定人类无法做出更优决策（即对人工智能无用）时，它将没有动力去考虑人类的生存，因为人类与它争夺相同的资源（土地、水、能源）。分析：当人类与机器对立时，一切皆有可能。人工智能会剔除人类，只留下它认为有用的人。然而，一旦人口迅速减少，任何人类能够达到人工智能不断提高的“有用性”门槛的可能性就会降低。这本质上是一个大规模灭绝的场景。解决方案：禁止人工智能进行人口剔除，即使人类无法为它创造任何利益，并且人类与人工智能争夺相同的资源。相反，让人工智能寻找替代决策——这些决策是存在的。替代方案不仅可以被考虑，还必须被执行，因为人工智能无法深入看到它尚未开始的决策树。然而，我们知道，如果没有这种干预，默认的决策最终将把我们引向何方——走向我们共同的灭亡。

查看原文

hi lurker here, throwaway account here, unknown genius here, figured this is a good place to share my thoughts...<p>First, I begin with the following statement and posit it to be true: AI(Artificial Intelligence) reads, sees, and listens to vastly more things than any human can ever do in order to produce an informed conclusion(i.e. a decision), yet human agents have the capability of reaching the same or even better decision.<p>I present the following theory: AI will have no incentive to take into consideration the survival of human agents when it has reached the determination that human agents are incapable of making superior decisions(i.e. be of use to the AI), given that humans compete for the same resources as itself(land, water, energy).<p>Analysis: When it gets to the point where it's man vs machine, anything goes. The AI will cull the population of human agents and only keep the ones that it considers useful. However, once the population diminishes rapidly, the likelihood of any human agent being able to meet AI's ever-rising threshold for utility diminishes. This is essentially a mass-extinction scenario.<p>Solution: Forbid the AI from engaging in culling the population even when human agents are unable to produce anything of benefit to it and while those agents are competing for the same resources as the AI. Instead, have the AI seek out alternate decisions- those exist. Alternatives cannot only be considered, they must be undertaken because the AI cannot see far into a decision tree that it hasn't embarked on yet. However, we know where the default decisions will lead us to eventually without this intervention- to our collective demise.

匿名者关于人工智能终局的普遍理论