Why did Google crash on Thursday, May 14th?
Those of us who depend heavily on Google and their related programs for our internet activity hit a brick wall on Thursday, as Google services (including search, Docs, Labs, and even Gmail) were slowed to a near standstill — and in some places stopped altogether.
Starting around 9:45 Central Standard Time on the morning of May 14, Google and all of its related services were the talk of the web. Blog posts appeared,wondering “What’s happened to Google?” In the United States, users from NYC, Chicago, Miami, and San Francisco (representing a nice cross section of the entire country) were complaining of the Google outage. Almost all users reported that Google searches were running at hilariously slow speeds, and that Gmail was all but dead. Similar reports have been rolling in from Europe and as far away as Australia.
Business website Forbes.com has suggested that one of the major Internet Service Providers could have altered its settings a bit, and inadvertently blocked many user’s traffic to the popular Google services. A spokesperson for Google, based in the US, was quick to insist that hackers were not to blame for the slowdown, but remained rather secretive about what was responsible. The US-based firm said the fault lasted around an hour and affected a “small percentage of Google users” — something we now know wasn’t entirely true.
“The issue affecting some Google services has been resolved,” said the spokesperson. “We’re sorry for the inconvenience, and we’ll share more details soon.”
Google’s search engine fields more than nine billion search requests every month in the United States alone. Because Google is so popular (it is estimated that hundreds of millions of people regularly use Google services) even a breakdown affecting a small percentage of its audience can appear to have a major impact.
To ensure it retains its current domination of the market, which in turn attracts massive ad revenue, Google has spent literally billions of dollars to create a major network of computers to decrease the chances of breakdowns like these. Consider their widespread network a sort of “safety net”.
The Internet Storm Center, a global security threat / internet status watchdog, is reporting that it received “multiple reports of a total fail of Google Applications. Gmail, Reader, Docs, News, Apps. etc.” There is still no official cause of this massive outage, leading many people into a hysterical state of conjecture.
The Storm Center went on to say that it “has reports of Google working in some areas” such as many parts of the American west coast, the UK, and a handful of others European nations. In America, and for much of the Western hemisphere, the outage does appear to have been pretty widespread. Whether or not you were personally affected likely depends on which Google Data Center you are connected to. While this Internet Storm Center report made it appear that Europe was mostly unaffected, we would learn later that many Google users in Europe were just as frustrated with the speed and availability of their Google functions.
There are a few ways to attempt a “diagnosis” of the problem. What the Internet Storm Center suggested, that the problem lie in Google’s Date Centers, may not be factual. After checking the website Just Ping, which “pings” remote hosts around the world to test their connection, web analysts determined that websites from all over the world were seeing as much as 50% web packet losses. This was notated at around 10:45 AM Central Standard Time. Web sites from Melbourne, Australia to Madrid, Spain were seeing their connections weakened and failing. It wasn’t just Google.
At this point, people were using words like “virus” and “Conficker worm”. As the hysteria mounted, some users found their Google services speeding up and coming back online. By noon on Thursday, many American users reported normal operation, but by then, the European Google crash was just starting to kick into high gear.
Before the blogger nerds came right out and suggested that this was the work of a hacker network, or of the infamous Conficker worm, there was one last diagnosis attempted. It was possible that Google’s problems were the result of trouble at the so called NOCs, or Network Operating Centers. The slowdowns and crashes could have been masking a bigger problem at Google — major network trouble on a global scale. Thankfully, the Internet Health Report didn’t show any significant network trouble, and as of this writing on Friday morning, there are still no serious network issues to be dealt with.
Another theory? The problem was due to AT&T’s NOC routing problems, probably suggested because AT&T is such a massive presence in the world of NOCs. This theory just doesn’t pass the smell test. Google outages were reported by a variety of users working with many different NOCs, many of whom reported network “trace routes” proving their internet connections never traveled on any AT&T networks.
After all of the initial frustration, panic began to set it. Bloggers and internet experts suggested that Google was undergoing a massive Distributed Denial of Service or DDoS attack.. Based on the size of the attack that would be needed to interfere with a network as massive as Google, many believed the slowdown was a direct result of an attack from the controllers of the Windows worm Conficker. This has not been verified.
While Google appears to be better in most of North America this morning, Google is still suffering serious network problems in the European Union and in South Africa.