An Insider's Look at DDoS Attacks

LOS ANGELES — For all webmasters and website owners, the prospect of suffering a distributed denial of service (DDoS) attack is a frightening prospect, posing performance, security, and customer service problems, along with a loss of revenue and reputation.

While many site operators might feel a DDoS attack is something their web host will handle, or that only happens to sites that “deserve it” due to their content, or as the result of competitive mischief, or a disgruntled party, the truth is your site can easily be swept up in a broader attack.

One recent example of a DDoS attack with a wide-ranging impact was the massive assault that occurred last Friday, October 21.

According to New Hampshire-based Dyn’s EVP Scott Hilton, this particular attack was “complex and sophisticated, using maliciously targeted, masked TCP and UDP traffic over port 53.”

Dyn manages traffic for major sites such as Netflix, Reddit, Spotify, and Twitter, and confirms the Mirai botnet was the primary source of malicious attack traffic — an attack compounded by recursive DNS retry traffic — in other words, folks hitting their browser’s “refresh” button, adding to the server loads.

As to the main culprit, Hilton says “Dyn is collaborating in an ongoing criminal investigation of the attack and will not speculate regarding the motivation or the identity of the attackers.”

“We began to see elevated bandwidth against our Managed DNS platform in the Asia Pacific, South America, Eastern Europe, and US-West regions that presented in a way typically associated with a DDoS attack. As we initiated our incident response protocols, the attack vector abruptly changed, honing in on our points of presence in the US-East region with high-volume floods of TCP and UDP packets, both with destination port 53 from a large number of source IP addresses,” Hilton explains. “The abrupt ramp-up time and multi-vectored nature of the attack, led to our Engineering and Network Operations teams deploying additional mitigation tactics on top of our automated response techniques.”

“These techniques included traffic-shaping incoming traffic, rebalancing of that traffic by manipulation of anycast policies, application of internal filtering and deployment of scrubbing services,” Hilton adds. “Mitigation efforts were fully deployed by 13:20 UTC; the attack subsided shortly after.”

Hilton notes that at roughly 15:50 UTC a second attack began.

“This attack was more globally diverse, but employed the same protocols as the first attack. Building upon the defenses deployed during the earlier attack and extending them globally, we were able to substantially recover from the second attack by 17:00 UTC. There was residual impact from additional sources that lasted until approximately 20:30 UTC,” Hilton says. “A number of probing smaller TCP attacks occurred over the next several hours and days; however, our mitigation efforts were able to prevent any further customer impact.”

Hilton explains that during a DDoS which uses the DNS protocol it can be difficult to distinguish legitimate traffic from attack traffic.

“For example, the impact of the attack generated a storm of legitimate retry activity as recursive servers attempted to refresh their caches, creating 10-20X normal traffic volume across a large number of IP addresses,” Hilton says. “When DNS traffic congestion occurs, legitimate retries can further contribute to traffic volume.”

Hilton notes that Dyn saw both attack and legitimate traffic coming from millions of IPs across all geographies.

“It appears the malicious attacks were sourced from at least one botnet, with the retry storm providing a false indicator of a significantly larger set of endpoints than we now know it to be,” Hilton explains. “We are still working on analyzing the data but the estimate at the time of this report is up to 100,000 malicious endpoints. We are able to confirm that a significant volume of attack traffic originated from Mirai-based botnets.”

Hilton says that early observations of the TCP attack volume from a few of Dyn’s datacenters indicate packet flow bursts 40 to 50 times higher than normal.

“This magnitude does not take into account a significant portion of traffic that never reached Dyn due to our own mitigation efforts as well as the mitigation of upstream providers,” Hilton explains. “There have been some reports of a magnitude in the 1.2 Tbps range; at this time we are unable to verify that claim.”

That is a huge amount of network-crushing bandwidth.

“This attack has opened up an important conversation about internet security and volatility,” Hilton concluded. “Not only has it highlighted vulnerabilities in the security of ‘Internet of Things’ (IOT) devices that need to be addressed, but it has also sparked further dialogue in the internet infrastructure community about the future of the internet.”

As they say, every dark cloud has a silver lining.