我的网站遭受的攻击比人类访问者还多。
2 分•作者: tommy2970•4 个月前
我在家里用树莓派 4B 运行了一个小型自托管网站。
几周前我开始思考：在 2026 年，到底是谁在访问网站？
不仅仅是人类。是所有一切。
于是，我基于 GoAccess 构建了一个公开的可观测性仪表板，将流量分为四类：人类访客、搜索引擎爬虫、AI 检索代理和自动化攻击。
过去 17 天的数据让我感到惊讶：
<p>4,523 人类访问
6,409 次自动化攻击尝试
来自搜索引擎和 AI 系统的数千次爬虫请求
<p>这些攻击并不复杂。它们大多是自动化扫描器，探测 .env 文件、WordPress 管理面板和云凭证——无论那里实际运行着什么，都会攻击互联网上的每一个公网 IP。
我发现更有趣的是 AI 代理的行为。
AI 检索代理（GPTBot、ClaudeBot、PerplexityBot、Amazonbot）的行为与传统的搜索爬虫不同。它们会积极地抓取语义文件——llms.txt、sitemap.xml、JSON-LD 结构化数据——并且似乎在索引网站的知识图谱结构，而不是单个页面。在新内容发布后的几小时内，多个 AI 爬虫已经访问过，这似乎是由站点地图更新触发的，而不是任何外部链接。
我发现了一些意想不到的观察：
<p>机器流量总和持续超过人类流量
AI 代理发现新内容的速度比 Google 快
网站暴露的语义结构似乎与内容本身同等重要
即使是住宅 ISP 上的树莓派也会收到持续的自动化扫描（平均每天 380 多次尝试）
<p>我公开了这个仪表板，因为我认为网络的机器端被低估了。
现代网络感觉不像“用户访问页面”，更像是一个并行的生态系统，爬虫、AI 代理和自动化系统与人类访客一起持续运行。
<p>有两个问题想问 HN：
是否有人将 AI 代理与传统搜索爬虫分开跟踪？
是否有人注意到 AI 检索系统索引语义结构（JSON-LD、llms.txt）的速度比索引页面内容快？
查看原文
I run a small self-hosted website on a Raspberry Pi 4B at home.
A few weeks ago I started wondering: who actually visits a website in 2026?
Not just humans. Everything.
So I built a public observability dashboard on top of GoAccess that separates traffic into four categories: human visitors, search engine crawlers, AI retrieval agents, and automated attacks.
The numbers from the last 17 days surprised me:<p>4,523 human visits
6,409 automated attack attempts
Thousands of crawler requests from search engines and AI systems<p>The attacks aren't sophisticated. They're mostly automated scanners probing for .env files, WordPress admin panels, and cloud credentials — hitting every public IP on the internet regardless of what's actually running there.
What I found more interesting was the AI agent behavior.
AI retrieval agents (GPTBot, ClaudeBot, PerplexityBot, Amazonbot) behave differently from traditional search crawlers. They hit semantic files aggressively — llms.txt, sitemap.xml, JSON-LD structured data — and seem to index the knowledge graph structure of a site rather than individual pages. Within hours of publishing new content, multiple AI crawlers had already visited, apparently triggered by the sitemap update rather than any external link.
A few observations I didn't expect:<p>Combined machine traffic consistently exceeds human traffic
AI agents discovered new content faster than Google did
The semantic structure exposed by the site seems almost as important as the content itself
Even a Pi on a residential ISP receives constant automated scans (380+ attempts/day average)<p>I made the dashboard public because I think the machine side of the web is underobserved.
The modern web feels less like "users visiting pages" and more like a parallel ecosystem of crawlers, AI agents, and automated systems running continuously alongside human visitors.<p>Two questions for HN:
Are others tracking AI agents separately from traditional search crawlers?
Has anyone else noticed AI retrieval systems indexing semantic structure (JSON-LD, llms.txt) faster than they index page content?