HackerNews中文版

我每天都使用 Claude Code，并在研究过程中测量了它在处理网页时的成本。例如，一篇普通的维基百科文章原始 HTML（使用 tiktoken 编码）有 68,240 个 token，而耐克官网首页则有 353,000 个 token。 Claude Code 内置的网页抓取功能可以很好地处理简单情况。它能将维基百科总结到大约 950 个 token，并且可以绕过 Indeed 和 Ticketmaster 等网站的 Cloudflare 防护。但是，（总会有个但是）对于 JavaScript 渲染的页面和一些反爬虫页面，它会返回空。 quotes.toscrape.com/js 会返回“未找到引文”；nike.com 会返回 403 错误。然后你的代理会将原始 HTML 重新放入上下文中，但仍然会失败。（注意：我也遇到过这种情况，在聊天结束时回顾发现它失败了，然后就从训练数据或来自其他来源的陈旧缓存中提取信息。）因此，我着手构建了一个开源的隐形浏览器（重新编译的 Chromium），它作为 Claude Code、Cursor 和 Claude Desktop 的 MCP 运行。本质上，我需要做的就是添加 MCP，它就能返回清理后的 token，同时还能规避检测：JavaScript 引文页面现在只需要 285 个 token，耐克官网只需要大约 700 个 token，而不是返回 403 错误。我仍在积极开发一些功能：目前还没有住宅出口节点，也无法绕过 Kasada 式的防护墙。它主要用于代理、QA 和研究。代码库和可复现的基准测试：https://github.com/tiliondev/fortress/tree/main/mcp 我是作者，欢迎大家提供反馈。

查看原文

i use claude code daily and measured what pages cost it while doing research. an average wikipedia article, for instance, is 68,240 tokens of raw html (tiktoken); nike's homepage is 353,000.claude code's built-in webfetch handles the easy case well. it summarizes wikipedia to about 950 tokens and clears cloudflare on some sites like indeed and ticketmaster. but, and there's always a but, on js-rendered and some anti-bot pages it returns nothing.quotes.toscrape.com/js gives "no quotes found"; nike.com gives a 403. your agent then dumps the raw html back into context and still fails. (note: i have also had cases where i read through the chat at the end and saw that it failed and just pulled from either training data or stale caches from other sources)so i worked on building an open-source stealth browser (recompiled chromium) that runs as an mcp for claude code, cursor, and claude desktop. essentially all i have to change form my end is add the mcp, and it returns the cleaned up tokens while also beating detection: the js quotes come back in 285 tokens, nike in about 700 instead of a 403.there is still stuff i am actively working on: there's no residential egress yet, and it won't beat kasada-style walls. it's for agents, qa, and research.repo and the reproducible benchmark: https://github.com/tiliondev/fortress/tree/main/mcp i'm the author and here for feedback.

一个维基百科页面会消耗您的 AI 代理 68,000 个 token。