Amazonbot Caught Hitting Fake Endpoints Planted in an LLM Honeypot, Despite an Explicit robots.txt Block

Hacker News·medium signalXBlueskyLinkedInCopy link

A Tell HN post describes a honeypot built earlier this year: a fake git repo containing fabricated HTTP endpoints, its address hidden only inside an HTML comment on a public page. Roughly three weeks ago Amazon Searchbot IPs began requesting the fake endpoints embedded in a shell script, even though robots.txt explicitly names Amazonbot. Commenters noted robots.txt carries no legal force and that ASN-level blocking against the published AWS ip-ranges.json is the practical remedy; one operator ranked Amazon fourth on their bad-scraper list. The transferable trick is the honeypot itself — planting unreachable endpoints is a cheap way to prove a crawler is parsing your content rather than merely fetching it.