Yesterday was a big outage on Level3's voice network nationwide. On any given day DownDetector is working overtime to track the numerous outages at banks, network operators, email, social networks and more.
This year I think every major carrier had an outage that lasted more than two hours. Some lasted all day. Some were regional. Some were nationwide.
Facebook, twitter, Gmail, Yahoo! mail, Outlook/Hotmail, Twitch, Slack, Playstation, Xbox Live - we live online. The new normal is not even 4 nines. Four 9's - 99.99% Uptime - is 1 hour of downtime per year. However, 99.9% means 8.76 hours of down time per year. We are in a 3-9's world.
It is too expensive to build a resilient network. And the carriers have learned that customers want cheap. With cable, they learned that Best Effort was "good enough". Who taught them that? Customers did. They buy unprotected circuits. They don't pay extra for protection or failover. BC/DR is for utilities. Even businesses in the Gulf region that survived Hurricane Katrina in 2005 can't be bothered with DR/BC. Not all. But most.
We only buy insurance when it is absolutely required by law.
Carriers also learned that a service level agreement (SLA) looked good to the buyer. It was a check box on the RFP. Yet the carriers know that paying out for an outage (just ask Frontier in FL, TX and Cali) isn't all that expensive. The most a customer can get back is a month's billing amount. The usual credit is 1/30 of the monthly bill. At $300 that is $10.
SD-WAN will help some businesses with back-up / failover, if they are willing to put a 4G or other broadband circuit in place as insurance. If they are willing to pay for the SD-WAN.
The funny thing is that no one thinks about Redundancy until there is an outage. Then they are peeved, yelling, calling it to be put on hold for hours. To be put on hold so they can vent and call back later to fight for a credit. Yippee!!
With all our data and applications in the cloud, Internet is as vital as oxygen or electricity. Down time means that no one is doing much of anything except Solitaire. How expensive is that?
Right now, Florida is staring into a category 4 hurricane named Matthew. Water, bread, gas, wood, batteries and other supplies are scarce. No one prepares. No one thinks ahead. Panic mode is just so unproductive.
Think about the cost of a 3 hour outage. No one does but it is our job as the Trusted Advisor to point at days like today and say: See?! Let's put a plan in place for this.
Disaster Recovery as a Service is a big deal especially for data center businesses. Right now, companies along the East Coast of the US are wishing they had planned better and bought some DR insurance.
Stay safe out there. Plan better starting tomorrow.
