Ask HN：如何阻止一个 AWS 机器人每月发送 20 亿次请求？
4 分•作者: lgats•8 个月前
我一直在与一个来自 AWS 新加坡的机器人——“Mozilla/5.0 (compatible; crawler)”——作斗争，它向我的一个域名发送了大量请求，几个月来平均每秒超过 700 个请求。
值得庆幸的是，CloudFlare 能够通过一个简单的 WAF 规则和 444 响应来处理流量，从而减少了出站流量。
我已经向 AWS 提交了几份投诉，要求停止这种流量，他们通常的回复是：
我们已经与我们的客户进行了沟通，并根据这次沟通确定，报告的活动目前不需要 AWS 采取进一步行动。
我尝试了各种 4XX 响应，看看机器人是否会停止，我也尝试了 30X 重定向（它会跟随），但都无济于事。
流量达到了需要我与 CloudFlare 重新协商合同的程度，并且在查看分析/日志时也造成了干扰。
我曾考虑将所有流量重定向到 aws 滥用报告页面，但以这种规模来看，它本质上是一个小型 DDoS 网络，将其发送到任何地方都可能被视为滥用行为。
其他人有类似的经历吗？
查看原文
I have been struggling with a bot– 'Mozilla/5.0 (compatible; crawler)' coming from AWS Singapore – and sending an absurd number of requests to a domain of mine, averaging over 700 requests/second for several months now.
Thankfully, CloudFlare is able to handle the traffic with a simple WAF rule and 444 response to reduce the outbound traffic.<p>I've submitted several complaints to AWS to get this traffic to stop, their typical followup is:
We have engaged with our customer, and based on this engagement have determined that the reported activity does not require further action from AWS at this time.<p>I've tried various 4XX responses to see if the bot will back off, I've tried 30X redirects (which it follows) to no avail.<p>The traffic is hitting numbers that require me to re-negotiate my contract with CloudFlare and is otherwise a nuisance when reviewing analytics/logs.<p>I've considered redirecting the entirety of the traffic to aws abuse report page, but at this scall, it's essentially a small DDoS network and sending it anywhere could be considered abuse in itself.<p>Are there others that have similar experience?