Sweden Declares War on My Web Server

Since 4 a.m. Friday, a computer at a Swedish IT company made more than 1.5 million web requests to my web site URouLette, which links to random web pages stored in a MySQL database. They're coming in at a speed of 38 requests a second. My MySQL database server can't handle that many requests, so by Friday afternoon Workbench and a bunch of other sites slowed to a crawl as the web server began belching black smoke. A massive crash was imminent.

The last time somebody did this, I used the Linux utility iptables to reject all connections from the offending IP address, which solved the problem easy peasy lemon squeezy. This time around, iptables failed with a "Can't open dependencies file" error.

My new friend in Sweden appears to be building a database of web addresses by requesting a URouLette script that loads a random web page over and over. This is both obnoxious and dumb -- all links on URouLette come from the Open Directory Project and can be downloaded in one file. I've reduced the severity of the problem by sending the same link with every request -- the company's home page.

Flooding a web server with this many requests constitutes a denial of service attack. In the time I've composed this blog entry, another 100,000 requests have been made. Ironically, an employee of the company blogged recently that it was suffering its own attack, though on a much larger scale:

Tens of thousands of machines on the internet suddenly started trying to access a single host within the network. The IP they targeted has in fact never been publicly used as long as we've owned it (which is just a bit under two years) and it has never had any public services.

We have no clue whatsoever why someone would do this against us. We don’t have any particular services that anyone would gain anything by killing. We're just very puzzled.

Our "ISP", the guys we buy bandwidth and related services from, said they used up about 1 gigabit/sec worth of bandwidth and with our "mere" 10megabit/sec connection it was of course impossible to offer any services while this was going on.

This is a good time to mention that I never liked Bjorn Borg.

Linux · Security · 2008/12/20 · 7 COMMENTS · Link

Comments

Once you get iptables working, you might want to consider tarpitting the source instead of just dropping traffic. Tarpitting consumes resources on the attacker's end and should slow them down.

iptables -t raw -A PREROUTING -p tcp -s bad.guys.ip.addr -j NOTRACK

iptables -I 1 INPUT -p tcp -s bad.guys.ip.addr -j TARPIT

(The NOTRACK rule prevents your own system from allocating resources unnecessarily.) If the traffic is coming from multiple source IPs, the address can also be a CIDR network range, e.g. 10.99.99.0/24.

Of course, if you can get in touch with the operator of the network in question and get them to behave, this won't be necessary.

#1 · Vance · 2008/12/21

so what was up with the iptables error, I've never had anything like that from it...

#2 · joe · 2008/12/21

On the specific error, you might try running "depmod" to rebuild the kernel module dependencies file, but without knowing your specific setup it's hard to tell whether this is likely to help or not.

#3 · Vance · 2008/12/22

I haven't found the cause yet -- it's some kind of conflict between the current kernel on the server and iptables.

#4 · Rogers Cadenhead · 2008/12/22

Maybe Rogers doesn't like Bjorn Borg but I know firsthand that he is gaa gaa over Abba. By the way Rogers you owe me a phone call, email or a million dollars - your choice.

#5 · LoneStar J.R. · 2008/12/28

Email sent. I think I still have my 45 of "Take a Chance on Me." Take a chance take a chance.

rachelmarsden.files.wordpress.com

#6 · Rogers CadenheaD · 2008/12/29

This is the explanation from the Swede responsible

daniel.haxx.se

#7 · Hillshum · 2008/12/29