LWN reports that the problem of AI scraper bots overwhelming websites has worsened since early 2025, with attacks traced primarily to residential proxy networks operated by both criminal and ostensibly legitimate entities. This escalating scraping activity threatens the sustainability of the open web by consuming server resources and undermining site owners' control over their content, with implications for content creators and web infrastructure operators. Attacks originate from millions of unique IP addresses via residential proxies, often without device owners' knowledge; the takedown of the IPIDEA botnet in early 2026 temporarily reduced traffic but the respite was short-lived.
Background
Residential proxies are IP addresses assigned by ISPs to real users, repurposed to route traffic for scraping. Attackers use these to appear as legitimate human visitors, evading simple IP-based blocking. Networks range from malware-compromised devices to 'ethically sourced' proxy services like Bright Data's free VPN.