AI 智能体测试
1 分•作者: dinasorous•6 个月前
在开发人工智能体（AI agents）的过程中，我一直在思考如何有效地测试它们。
随着我们整合更多的知识来源并扩展智能体的能力，测试变得越来越复杂。作为标准做法，我们使用评估（evals）来确保质量得到维持。但说实话，我感觉还缺少些什么。
我遇到的问题是，作为工程师，我们有时缺乏足够的领域知识来准确评估智能体的回应。与此同时，现有的工具限制了我们与领域专家合作进行共同测试的可能性。例如，目前的工具更侧重于仪表盘，而忽略了实际结果的可读性。
这就是我到目前为止的经验——我很想听听你们对此的看法。
查看原文
As I work on AI agents, I find myself constantly thinking about how to effectively test them.
As we integrate more knowledge sources and expand our agents' capabilities, testing becomes increasingly complex. As a standard practice, we use evals to ensure quality is maintained. But honestly, I feel like something is missing.
The issue I’m seeing is that we, as engineers, sometimes lack sufficient domain knowledge to assess an agent's response accurately. At the same time, current tooling limits the possibility of collaborating with domain experts to perform testing together. For example, current tooling gives priority to dashboards over the readability of actual outcomes
This has been my experience so far—I would love to hear your thoughts on this.