Show HN: Sentience – AI Agent的语义视觉定位（WASM 和 ONNX）
1 分•作者: tonyww•6 个月前
大家好，我是 SentienceAPI 的独立创始人。过去几个月，我一直在构建一个专为 LLM 智能体设计的浏览器自动化运行时。<p>问题：构建可靠的 Web 智能体非常痛苦。本质上有两种糟糕的选择：<p>原始 DOM：转储 document.body.innerHTML 便宜/快速，但会淹没上下文窗口（10 万+ tokens），并且缺乏空间上下文（智能体尝试点击隐藏或屏幕外的元素）。
视觉模型（GPT-4o）：发送截图很稳健，但速度慢（3-10 秒延迟）且昂贵（约 0.01 美元/步）。更糟糕的是，它们经常会产生坐标幻觉，导致按钮偏差 10 像素。<p>解决方案：语义几何 Sentience 是智能体的“视觉皮层”。它位于浏览器和您的 LLM 之间，将嘈杂的网站转化为干净、排名、坐标感知的 JSON。<p>工作原理（技术栈）：<p>客户端（WASM）：一个 Chrome 扩展程序注入一个 Rust/WASM 模块，该模块直接在浏览器进程中修剪 95% 的 DOM（脚本、跟踪像素、不可见的包装器）。它在 50 毫秒内处理 Shadow DOM、嵌套的 iframe（“帧拼接”）和计算样式（可见性/z-index）。<p>网关（Rust/Axum）：修剪后的树被发送到 Rust 网关，该网关使用简单的视觉提示（例如 is_primary）应用启发式重要性评分。<p>大脑（ONNX）：一个服务器端 ML 层（通过 ort 运行 ms-marco-MiniLM）根据用户目标（例如，“搜索鞋子”）对元素进行语义重新排序。<p>结果：您的智能体获得一个包含前 50 个最相关的可交互元素列表，其中包含精确的 (x,y) 坐标、重要性值和视觉提示，帮助 LLM 智能体做出决策。<p>性能：<p>成本：约 0.001 美元/步（而视觉模型为 0.01 美元+）
延迟：约 400 毫秒（而视觉模型为 5 秒+）
有效载荷：约 1400 tokens（而原始 HTML 为 10 万）
开发者体验（“酷”的东西）：我讨厌调试文本日志，所以我构建了 Sentience Studio，一个“时光旅行调试器”。它将每一步（DOM 快照 + 截图）记录到 .jsonl 轨迹中。您可以像视频编辑器一样浏览时间线，准确地查看智能体看到了什么，以及它产生了什么幻觉。<p>链接：<p>文档和 SDK：<a href="https://www.sentienceapi.com/docs" rel="nofollow">https://www.sentienceapi.com/docs</a><p>GitHub (SDK):
SDK Python: <a href="https://github.com/SentienceAPI/sentience-python" rel="nofollow">https://github.com/SentienceAPI/sentience-python</a><p>SDK TypeScript: <a href="https://github.com/SentienceAPI/sentience-ts" rel="nofollow">https://github.com/SentienceAPI/sentience-ts</a><p>Studio 演示：<a href="https://www.sentienceapi.com/docs/studio" rel="nofollow">https://www.sentienceapi.com/docs/studio</a><p>构建 Web 智能体：<a href="https://www.sentienceapi.com/docs/sdk/agent-quick-start" rel="nofollow">https://www.sentienceapi.com/docs/sdk/agent-quick-start</a><p>带有重要性标签的截图（金星）：
<a href="https://sentience-screenshots.sfo3.cdn.digitaloceanspaces.com/Screenshot" rel="nofollow">https://sentience-screenshots.sfo3.cdn.digitaloceanspaces.co...</a> 2026-01-06 at 7.19.41 AM.png<p><a href="https://sentience-screenshots.sfo3.cdn.digitaloceanspaces.com/Screenshot" rel="nofollow">https://sentience-screenshots.sfo3.cdn.digitaloceanspaces.co...</a> 2026-01-06 at 7.19.41 AM.png<p>我正在用 Rust 处理后端，用 Python/TypeScript 处理 SDK。该项目目前处于 beta 发布阶段，我非常欢迎大家对架构或排名逻辑提出反馈！
查看原文
Hi HN, I’m the solo founder behind SentienceAPI. I’ve spent the last December building a browser automation runtime designed specifically for LLM agents.<p>The Problem: Building reliable web agents is painful. You essentially have two bad choices:<p>Raw DOM: Dumping document.body.innerHTML is cheap/fast but overwhelms the context window (100k+ tokens) and lacks spatial context (agents try to click hidden or off-screen elements).
Vision Models (GPT-4o): Sending screenshots is robust but slow (3-10s latency) and expensive (~$0.01/step). Worse, they often hallucinate coordinates, missing buttons by 10 pixels.
The Solution: Semantic Geometry Sentience is a "Visual Cortex" for agents. It sits between the browser and your LLM, turning noisy websites into clean, ranked, coordinate-aware JSON.<p>How it works (The Stack):<p>Client (WASM): A Chrome Extension injects a Rust/WASM module that prunes 95% of the DOM (scripts, tracking pixels, invisible wrappers) directly in the browser process. It handles Shadow DOM, nested iframes ("Frame Stitching"), and computed styles (visibility/z-index) in <50ms.<p>Gateway (Rust/Axum): The pruned tree is sent to a Rust gateway that applies heuristic importance scoring with simple visual cues (e.g. is_primary)<p>Brain (ONNX): A server-side ML layer (running ms-marco-MiniLM via ort) semantically re-ranks the elements based on the user’s goal (e.g., "Search for shoes").<p>Result: Your agent gets a list of the Top 50 most relevant interactable elements with exact (x,y) coordinates with importance value and visual cues, helping LLM agent make decision.<p>Performance:<p>Cost: ~$0.001 per step (vs. $0.01+ for Vision)
Latency: ~400ms (vs. 5s+ for Vision)
Payload: ~1400 tokens (vs. 100k for Raw HTML)
Developer Experience (The "Cool" Stuff): I hated debugging text logs, so I built Sentience Studio, a "Time-Travel Debugger." It records every step (DOM snapshot + Screenshot) into a .jsonl trace. You can scrub through the timeline like a video editor to see exactly what the agent saw vs. what it hallucinated.<p>Links:<p>Docs & SDK: <a href="https://www.sentienceapi.com/docs" rel="nofollow">https://www.sentienceapi.com/docs</a><p>GitHub (SDK):
SDK Python: <a href="https://github.com/SentienceAPI/sentience-python" rel="nofollow">https://github.com/SentienceAPI/sentience-python</a><p>SDK TypeScript: <a href="https://github.com/SentienceAPI/sentience-ts" rel="nofollow">https://github.com/SentienceAPI/sentience-ts</a><p>Studio Demo: <a href="https://www.sentienceapi.com/docs/studio" rel="nofollow">https://www.sentienceapi.com/docs/studio</a><p>Build Web Agent: <a href="https://www.sentienceapi.com/docs/sdk/agent-quick-start" rel="nofollow">https://www.sentienceapi.com/docs/sdk/agent-quick-start</a><p>Screenshots with importance labels (gold stars):
<a href="https://sentience-screenshots.sfo3.cdn.digitaloceanspaces.com/Screenshot" rel="nofollow">https://sentience-screenshots.sfo3.cdn.digitaloceanspaces.co...</a> 2026-01-06 at 7.19.41 AM.png<p><a href="https://sentience-screenshots.sfo3.cdn.digitaloceanspaces.com/Screenshot" rel="nofollow">https://sentience-screenshots.sfo3.cdn.digitaloceanspaces.co...</a> 2026-01-06 at 7.19.41 AM.png<p>I’m handling the backend in Rust and the SDKs in Python/TypeScript. The project is now in beta launch, I would love feedbacks on the architecture or the ranking logic!