As some of my closest friends will attest, around 4 PM yesterday I was in overdrive over Community Matters. With no apparent explanation, the screen went dark and Community Matters disappeared from the Internet. Convinced it was my computer, I madly called friends only to discover that no one could find Community Matters online.
A very calm ‘computer’ friend assured me that it was probably a Word Press issue, and had nothing to do with Community Matters. But I was convinced that there were people who had hacked in to the system to assure that Community Matters was offline permanently. The funny thing is that it wasn’t just me . . . I had a number of Community Matter junkies who are dedicated followers of the blog that were likewise convinced that someone had done serious damage to my site somehow. One friend actually tracked down an administrator at Word Press and was assured that it was a network-wide problem, not just Community Matters. All I could think of was all the work that I had done and all the comments from readers . . . would we ever be up and running again?
Within a couple of hours the Word Press system was repaired and Community Matters was back online. To those who watched it disappear and were concerned, thank you! I guess it was just a bit of over-reaction on my part.
I received the following explanation from Word Press who hosts Community Matters, and thought you might find their explanation of interest. If all life’s problems could be so easily solved!
WordPress.com Downtime Summary
WordPress.com was down yesterday for approximately 110 minutes, our worst downtime in four years. The outage affected 10.2 million blogs, including our VIPs, and appears to have deprived those blogs of about 5.5 million pageviews.
What Happened: We are still gathering details, but it appears an unscheduled change to a core router by one of our datacenter providers messed up our network in a way we haven’t experienced before, and broke the site. It also broke all the mechanisms for failover between our locations in San Antonio and Chicago. All of your data was safe and secure, we just couldn’t serve it.
What we’re doing: We need to dig deeper and find out exactly what happened, why, and how to recover more gracefully next time and isolate problems like this so they don’t affect our other locations. I will update this post as we find out more, and have a more concrete plan for the future.
I know this sucked for you guys as much as it did for us — the entire team was on pins and needles trying to get your blogs back as soon as possible. I hope it will be much longer than four years before we face a problem like this again.