If you have ever tailed your nginx logs and wondered why your site gets more attention from random servers in Singapore than from actual humans, congratulations: you have met the Internet.
The other night I was just checking in on my fail2ban.log like any good admin does, half expecting peace and quiet. Instead, I was greeted by a stream of blocked logins, 404s, and connection attempts from IPs all over the planet. I sat there watching it scroll by, thinking: if this is just my little corner of the web, how much of the entire Internet is doing the same dance right now?
Every second of every day, some bot is knocking on your digital door, hoping you will let them in to steal, scrape, or spam something. And when you multiply that across the entire Internet, it starts to look a lot less like a network of people sharing information and a lot more like a junkyard of scripts yelling “Are you vulnerable?” at each other.
The depressing math of Internet “traffic”
Akamai says 42% of all web traffic in 2024 came from bots, and about two thirds of that were the bad kind. SOAX called it “the highest share of automated traffic in Internet history,” landing at almost half of everything online. Thales Group backed that up too, reporting that nearly half of the world’s Internet traffic last year was not human at all.
Even Imperva, who has tracked this for years, said malicious bots alone made up around 31% of total traffic and were growing faster than the “good” ones like search crawlers or uptime monitors. Akamai also reported a 33% year-over-year jump in overall bot activity. So if you picture the global Internet as a four-lane highway, only two lanes have actual drivers. The rest are packed with self-driving malware bumping into each other and occasionally trying to steal your hubcaps.
Not all bots are evil… just most of them
Sure, some bots are useful. Google’s crawler, uptime monitors, maybe that RSS fetcher that still remembers what the web was supposed to be. But most of what hits your server these days is garbage. Probes from scanners looking for a forgotten phpMyAdmin install from 2008. Headless Chrome instances trying to harvest email addresses. Spam bots posting “Nice blog! Check my crypto tips!” on every comment form in existence.
They do not pay bandwidth bills, they do not read your content, and they definitely do not click your affiliate links. They just waste your time, your CPU cycles, and your logs.
The cost of all this “activity”
Globally, that translates to an absurd amount of wasted bandwidth. Petabytes, easy. CDNs, ISPs, and data centers are moving mountains of packets that serve absolutely no one. Entire racks hum just to drop connections. Millions of dollars in infrastructure exist purely to say “no” faster.
It is like building bigger highways not because traffic increased, but because the same ten thousand people keep driving in circles.
Even at the small end, every self-hosted admin knows the feeling: fail2ban filling your logs, nftables doing squats all night, and your poor server muttering “I swear, if one more bot asks for /wp-login.php…”
What it says about the modern web
The Internet used to be about connection. Now half of it is about defense. Half of all bytes sent across the planet are basically background radiation from lazy attacks and junk automation. It is no wonder everyone hides behind CDNs and reverse proxies because the open web is under constant siege by toaster scripts.
Yet in a weird way, it is kind of poetic. Our servers have evolved like immune systems. Fail2ban, CrowdSec, WAFs, they are the antibodies of a web that refuses to die no matter how much nonsense floods it.
Where the big players fit in
The big clouds are in the same war, just at a bigger scale. Cloudflare, Akamai, and Google spend fortunes filtering this junk before it ever hits your port 80. Cloudflare alone blocks hundreds of billions of bad requests every single day. Akamai says their edge network mitigated 311 billion web attacks last year. Even AWS now has whole services dedicated to fighting bots, from WAF rule engines to full-on bot management suites.
It is an arms race. Every time someone builds a better filter, the botnets get smarter, cheaper, and more distributed. And while Big Tech fights it at the CDN layer, the rest of us are out here at the edge holding the line one fail2ban rule at a time.
But honestly, that is part of the charm. The Internet still works because a bunch of stubborn sysadmins refuse to let it rot completely.
So what do we do?
We keep doing what sysadmins do best: block, patch, laugh, repeat. We tune our firewalls, rotate our keys, and rate limit like it is religion. Because somewhere in that noise, an actual person might still show up to read what we wrote.
Half the world’s bandwidth might be wasted on bots, but the other half is still ours. That is the part powered by curiosity, chaos, and caffeine. And as long as we keep the packets flowing, there is still a little magic left in the wires.