我搭建了一个小型实验,用于检测 AI 编程助手何时出现偏差。

1作者: brainchen5 个月前
最近在使用 Cursor 编写代码时,我遇到了一个常见问题。 通常,我们一开始会给 AI 一些明确的约束,例如: * 不要修改数据库模式 * 不要修改某些 API * 只允许修改前端逻辑 * 某些函数名不能更改 在对话开始时,AI 通常能很好地遵守这些规则。 然而,随着对话的进行,例如达到 4 万或 5 万个 tokens 时,一个常见的问题就出现了: AI 会逐渐“忘记”之前提到的限制。 例如: 一开始你说“不要修改数据库”, 但几轮之后,它突然建议: “我们可以通过修改数据库结构来解决这个问题。” 我之前问过别人如何解决这个问题,有人建议: 在项目中编写一个 important.md 或规则文件,让 AI 每次都读取它。 这种方法确实有一些效果,但在实际开发中仍然会出现问题。 例如: 一开始你说“不要动数据库 A”, 但后来项目中添加了数据库 B。 如果你没有及时更新 markdown 文件,AI 可能会意外地修改你不想更改的内容。 所以我最近创建了一个小型实验工具,主要用于解决 AI 编程助手中的“约束漂移”问题。
查看原文
I recently encountered a common problem while coding with Cursor.<p>Initially, we usually give the AI some explicit constraints, such as:<p>Don&#x27;t modify the database schema<p>Don&#x27;t modify certain APIs<p>Only allow modifications to the front-end logic<p>Some function names cannot be changed<p>At the beginning of the conversation, the AI generally follows these rules well.<p>However, as the conversation lengthens, for example, to 40,000 or 50,000 tokens, a common problem arises:<p>The AI gradually &quot;forgets&quot; the previously mentioned restrictions.<p>For example:<p>Initially you say &quot;Don&#x27;t modify the database,&quot;<p>but after a few rounds, it suddenly suggests:<p>&quot;We can solve this problem by modifying the database structure.&quot;<p>I&#x27;ve asked others how to solve this before, and some suggested:<p>Write an important.md or rule file in the project so the AI reads it every time.<p>This method does have some effect, but problems still arise in actual development.<p>For example: Initially you say &quot;Don&#x27;t touch database A,&quot;<p>but later database B is added to the project.<p>If you don&#x27;t update the markdown file in time, the AI might accidentally modify things you didn&#x27;t intend to change.<p>So I recently created a small experimental tool, mainly to solve the &quot;constraint drift&quot; problem in AI programming assistants.