Show HN: Τ³-Bench 发布 – 智能体能处理复杂文档和实时通话吗？
5 分•作者: victorbarres•5 个月前
τ-Bench 是一个开放基准，用于评估 AI 智能体在有可验证结果的、基于现实世界的、多轮客户服务任务上的表现。自发布以来，看到社区采用它，我们感到非常高兴——这是第三次迭代。通过 τ³-Bench，我们将其扩展到两个新的场景：知识密集型检索和全双工语音。
τ-Knowledge：智能体必须浏览约 700 份相互关联的政策文件，以完成多步骤任务。最佳前沿模型（GPT-5.2，高推理能力）的完成率约为 25%。令人惊讶的是：即使你向模型提供了它所需的精确文件，性能也仅达到约 40%。我们发现瓶颈不在于检索——而在于对复杂、相互关联的政策进行推理，并按正确的顺序执行正确的操作。
τ-Voice：相同的基于现实世界的任务，但通过具有真实音频的实时全双工语音进行——包括口音、背景噪音、中断、压缩电话线路。在清晰音频条件下，语音智能体的得分是 31–51%，在真实条件下是 26–38%。一个在不同提供商（OpenAI、Gemini、xAI）中都存在的失败模式是：智能体在身份验证过程中误听了姓名或电子邮件，导致后续所有操作失败。
我们还在原始的航空公司、零售和电信领域中加入了 75+ 项任务修复——其中许多是基于社区审计和 PR（包括来自亚马逊和 Anthropic 的贡献）。我们认为基准测试的质量取决于其维护，我们感谢社区对改进它的帮助。
代码和排行榜是开放的——我们欢迎社区提交和反馈。
博客文章（论文、代码、排行榜）：[https://sierra.ai/blog/bench-advancing-agent-benchmarking-to-knowledge-and-voice](https://sierra.ai/blog/bench-advancing-agent-benchmarking-to-knowledge-and-voice)
查看原文
τ-Bench is an open benchmark for evaluating AI agents on grounded, multi-turn customer service tasks with verifiable outcomes. It's been great to see the community adopt it since launch — this is now the third iteration. With τ³-Bench, we're extending it to two new settings: knowledge-intensive retrieval and full-duplex voice.<p>τ-Knowledge: agents must navigate ~700 interconnected policy documents to complete multi-step tasks. Best frontier model (GPT-5.2, high reasoning) hits ~25%. The surprising part: even when you hand the model the exact documents it needs, performance only reaches ~40%. We found that the bottleneck isn't retrieval — it's reasoning over complex, interlinked policies and executing the right actions in the right order.<p>τ-Voice: same grounded tasks, but over live full-duplex voice with realistic audio — accents, background noise, interruptions, compressed phone lines. Voice agents score 31–51% in clean audio conditions and 26–38% in realistic ones. A consistent failure pattern across providers (OpenAI, Gemini, xAI): agent mishears a name or email during authentication, and everything downstream fails.<p>We also incorporated 75+ task fixes to the original airline, retail, and telecom domains — many based on community audits and PRs (including contributions from Amazon and Anthropic). We believe a benchmark is only as good as its maintenance, and we're grateful for the community's help improving it.<p>Code and leaderboard are open — we'd welcome community submissions and feedback.<p>Blog post (papers, code, leaderboard): <a href="https://sierra.ai/blog/bench-advancing-agent-benchmarking-to-knowledge-and-voice" rel="nofollow">https://sierra.ai/blog/bench-advancing-agent-benchmarking-to...</a>