展示 HN：为您的 AI 代理提供屏幕指南，指示用户点击何处
7 分•作者: pancomplex•17 天前
各位 HN 的朋友们，我是 Christian，Frigade（YC W23）的联合创始人之一。我注意到许多应用内 AI 助手在实际理解它们所处的产品的能力上存在不足。
举个例子，假设用户在一个特定的 SaaS 产品中询问如何完成某项操作。理想情况下，AI 助手可能会回复说它有一个工具可以完成这项任务，并直接为用户自动化该过程。这是一个很棒的结果。
但很多时候并非如此。可能没有针对该特定任务的工具调用，或者用户的疑问最好通过特定的 UI 工作流程或界面来解决。在这些情况下，许多 AI 助手往往会回退到对帮助中心进行基本的 RAG（检索增强生成），有时甚至会搜索互联网来理解自己的产品。这可能是一个非常缓慢的过程，而且大多数时候，帮助中心的文章因为产品更新速度快于文档而过时。更糟糕的是，没有人喜欢阅读一长串的要点，然后将其映射到 UI 上。
我的工具（Assist API）通过一个简单的工具调用解决了这个差距，该调用定义如下：
```javascript
const frigade_guide_tool = {
description: '调用此工具来回答产品问题或引导用户完成任务。',
parameters: {
query: {
type: 'string',
description: '用户正在询问或想要做什么',
},
},
run: ({ query }) => frigade.assist({ query }),
}
```
这是我录制的关于如何使用 Vercel AI SDK 进行设置的演示：https://www.youtube.com/watch?v=9WQ0UbLjC6I
当调用此工具时，它将执行以下操作：
1. 收集用户屏幕上显示的内容、用户权限、功能标志等上下文信息。然后执行以下操作之一：
2a. 如果可解决：生成一个屏幕上的指南，说明如何解决某个问题。
2b. 如果是概念性的：返回文本，向父代理描述如何解决问题。
2c. 拒绝（即无法提供帮助）。
这个工具是如何知道该做什么的？
该工具通过一个基于浏览器的代理来学习给定应用程序的 UI。您为您的软件提供一个测试账户（例如，暂存或预览环境），一个浏览器代理会登录并遍历整个产品。然后，它会构建一个关于应用程序如何工作的内部地图，该地图可以用于查询任何与产品相关的问题，或如何从 UI 中的 A 点导航到 B 点。它还会根据这张地图编写自己的文档。该代理会按计划重新运行，或者可以通过 CI/CD 触发。
文档和更多详细信息：https://frigade.com/assist-api
查看原文
Hey HN. I'm Christian, one of the founders of Frigade (YC W23). I've noticed that a lot of in-app AI agents struggle to actually understand the products they exist in.<p>For instance, let's say a user asks an agent how to do something in a given SaaS product. In an ideal case, maybe that agent replies saying it has a tool to do the task and just automates that work entirely for the user. That's a great outcome.<p>But often that's not the case. Maybe there is no tool call for that exact task, or maybe the user's question is best solved by a specific UI workflow or interface. In these cases, many agents tend to fall back on basic RAG on their help center, or sometimes even searching the internet for an understanding of their own product. This can be a very slow process and most of the time help center articles are outdated as products evolve faster than them today. Even worse, no one likes reading a long list of bullets and mapping that back to a UI.<p>My tool (Assist API) solves this gap with a single tool call defined like this:<p><pre><code> const frigade_guide_tool = {
description: 'Call this tool to answer product questions or guide the user through a task.',
parameters: {
query: {
type: 'string',
description: 'What the user is asking or wants to do',
},
},
run: ({ query }) => frigade.assist({ query }),
}
</code></pre>
Here's a demo I recorded on how to set it up with the Vercel AI SDK: <a href="https://www.youtube.com/watch?v=9WQ0UbLjC6I" rel="nofollow">https://www.youtube.com/watch?v=9WQ0UbLjC6I</a><p>When called, the tool will do the following:<p>1) Gather context on what the user is seeing on screen, their permissions, feature flags, and more. Then one of the following:<p>2a) If solvable: Generate an on screen guide for how to fix a given problem<p>2b) If conceptual: Return text describing to the parent agent how to solve the problem<p>2c) Reject (i.e. unable to help)<p>How does the tool know what to do?<p>The tool learns a given application UI by using a browser-based agent. You provide a test account to your software (i.e. staging og preview), and a browser agent logs in and works its way through the entire product. It then builds its own map of how the application works which can the be queried about any product-related question or how to get from A to B in the UI. It also writes its own documentation from this map. The agent re-runs on a schedule or can be triggered through CI/CD.<p>Docs and more details: <a href="https://frigade.com/assist-api">https://frigade.com/assist-api</a>