I’ve just attended a one day “seminar” with folks at Arbor Networks and it has been insightful.

It seems people are still pretty ignorant about DDoS attacks. Unlike the 1999 CIH virus that was programmed to take out a computer by corrupting it’s BIOS EEPROM, most of the viruses, worms, malwares and whatnots on the Internet today are around for one simple reason – money. Obviously if you’re good enough to write worms, you’d think “why write a worm for fun, when I can make money?” These worms infect computers to build Botnets, and Botnets are sold for real money on the black market to take down sites (via a DDoS), send spam, and all sorts of other things.

There was one point in particular though that caught my attention, and it was that firewalls (or in fact any type of inline device such as load balancers) are potentially targets for DDoS attacks. To make matters worse, the higher the OSI layer the firewall capability goes, the worse it gets in terms of performance and reliability.

Believe it or not, firewalls are vulnerable to serious security issues like buffer overflows just like any other server or appliance with an IP address. So it turns out that firewalls are the biggest marketing scam in the history of IT security because companies have spent millions and millions of dollars on these stuff that don’t offer much protection than say, iptables.

Just about a month ago, I spoke to one of our customers who experienced a DDoS attack launched towards their co-location in the USA. The DDoS traffic was approximately 500Mbps and it completely took out the firewall. This site provided online payment services to customers and was up and down for days. Their firewall was tiny in comparison to the DDoS they got – on paper specs states performance capabilities of 90Mbps or 30Kpps at 2.8K sessions/sec with a max of 8K sessions at any time. Of course, these are lab specifications and real world traffic wouldn’t be as forgiving.

A simple DDoS attack that’s merely 10Mbps in traffic volume would have generated millions of packets per second with a 1-byte  UDP or ICMP packet. Taking down such a firewall would be a breeze. In fact, a single modern day computer on a broadband connection could probably do the job.

If it was a TCP SYN flood, it would have been way easier. Sending 2K TCP SYN packets per second is child’s play, so filling the firewall’s state table really takes no more than 10 seconds.

I had a chat with my wife who audits financial institutions (FIs) based on the PCI-DSS standard. Most FIs providing payment card services will have to conform to this standard. This standard, however, mandates that a firewall is required to comply. Unfortunately, most FIs have a pretty average Internet connectivity pipe – somewhat in the range of 20Mbps to 100Mbps. They scale their firewalls to their connectivity, so what they have, well, closely resembles the one I described earlier.

So why were firewalls invented?

Early operating systems didn’t provide packet filtering capabilities, so the early firewalls were really just stateless packet filters that basically routed (not NAT’ed) traffic and dropped unwanted requests based on simple IP, protocol and port numbers to services that weren’t supposed to be public. Then the idea of NAT came about (remember the days of WinRoute) to allow multiple computers on a LAN to share a single IP address on a WAN link. Some smart guy then figured, “oh well, let’s put servers on a private subnet and use the NAT technology to map public and private address spaces. This way, we’re safer!” Agreeably, that was the dumbest idea ever and is a PITA to manage, but millions of servers are configured this way today. Over time, these features were slowly incorporated into the all-in-one junkbox we now call the Firewall. Sweet.

Personally, I don’t have a firewall sitting in front of my servers. All my servers are individually configured to run iptables (or ipfilter on Solaris, etc.). I am going to test the Linux TCP stack with Apache from a default CentOS install to see how much SYN flood it can hold up before giving up and maybe post some results here, including what I tweaked in the kernel.