Bots are taking over the web, according to Cloudflare CEO Matthew Prince. In an interview at the SXSW conference in Austin this week, he said that with the speed at which artificial intelligence is growing, AI bot traffic will exceed the amount of human traffic that’s online by 2027.
Prince explained that bots’ web usage has been increasing alongside the growth of generative AI technology because bots are capable of visiting far more sites to get answers for users’ chatbot queries.
“If a human were doing a task — let’s say you were shopping for a digital camera — and you might go to five websites. Your agent or the bot that’s doing that will often go to 1,000 times the number of sites that an actual human would visit,” Prince said. “So it might go to 5,000 sites. And that’s real traffic, and that’s real load, which everyone is having to deal with and take into account.”
Before the generative AI era, the internet was only about 20% bot traffic, with Google’s web crawler being the largest, according to Prince, whose infrastructure and security company is used by one-fifth of all websites. But beyond some other reputable crawlers, the only other bots were those used by scammers and bad actors.
“With the rise of generative AI, and its just insatiable need for data, we’re seeing a rise where we suspect that, in 2027, the amount of bot traffic online will exceed the amount of human traffic that’s online,” Prince said.
The executive also noted that this change to the web would require the development of new technologies, like sandboxes for AI agents that can be spun up on the fly and then torn down when their task has finished. These could come into play when consumers ask AI agents to perform certain tasks on their be …