10 Ways to Make Your Outage Emergency Room Fun

Originally published in Devops Digest website. By Andrew Marshall, Director of Product Marketing at Cedexis

Blogpost Make outages fun

It’s 3:47am. You and the rest of the Ops team have been summoned from your peaceful slumber to mitigate an application delivery outage. As you catch up on the frantic emails, Slack chats, and text messages from everyone on your international sales team (Every. Single. One.), your mind races as you switch to problem solving mode. It’s time to start thinking about how to make this mitigation FUN!

1. GENTLY REMIND EVERYONE THAT THE SAAS-DELIVERED AUTOMATED APPLICATION DELIVERY STRATEGY YOU PROPOSED WOULD HAVE PREVENTED THIS

No need to rub it in, but … if you had turned on a software-defined application delivery platform for your hybrid infrastructure, you’d all be sound asleep right now. Automated real-time delivery decisions and failover would be nice, right? Just sayin’.

2. DROP A GENERATIONALLY-DIVISIVE GIPHY IN THE EMERGENCY SLACK CHANNEL

Your coworkers’ opinions on gaming consoles, Spiderman movies, and music are different than yours! The perfect animated GIF will remind them of your erudite tastes, while you have their attention. Extra credit if you can work in some low-key shade about not listening to your equally sophisticated opinions on optimized outage mitigation.

3. HOLD AN IMPROMPTU CLOUD HEALTH DASHBOARD PAGE CUSTOMIZATION CONTEST

Oh hey, look at that! Your cloud provider’s health dashboard page says everything is fine…because it’s powered by the services that went down. Help your team vent their creative energy (and frustration) with some fun customized MS Paint updates to the offending page. Bonus points for art that reminds everyone of the value of a multi-cloud strategy powered by a programmable application delivery platform.

4. USE HEALTH METRICS TO CURE ILLS INSTEAD OF JUST CONDUCTING ANOTHER REALLY GOOD POST-MORTEM

Tired of “learning lessons” from these emergency room drills? You depend on NGINX for your LLB, but don’t have a way to use those LLB health metrics and data to automate global delivery. Disjointed delivery intelligence means you don’t know how your apps will land with users. You need an end user-centric approach to app delivery that automates the best delivery path and ensures failover is in place at all times. Micro-outages often fly under your passive monitoring radar, but that doesn’t mean your users don’t notice them. An active, integrated app delivery approach re-routes automatically before you lose business. Post-mortems are fun…but so is making sure your apps survive the last mile. Arguably more so?

5. OPEN (AND THEN CLOSE) SOME TICKETS FOR THE SALES TEAM

“Sales needs to sell more stuff.” You’ll feel better.

6. BUILD A COUNTDOWN CLOCK FOR WHEN YOU CAN FINALLY REPLACE YOUR OLD ADC HARDWARE

Sure, your Mode1 ADC hardware was a sunk cost so you’re stuck with them for a while. But you’re one unnecessary emergency closer to having a fully software-defined application delivery platform for your hybrid cloud. And now you’re even closer. And closer … Tick. Tock.

7. LEVERAGE REAL USER MEASUREMENTS (RUM) TO FORESEE AND FIX PROBLEMS BEFORE THEY HAPPEN

Probably best to do this during normal work hours. User experience data from around the world can detect degrading, sluggish resources in real-time, and user-centric app delivery logic powered by RUM can make quick re-routing decisions automatically. No more getting woken up after the application crashes. While you’re wishing you had RUM on your side, you can look up some fun facts from the countries experiencing app outages. Did you know Luxembourgish is an official language?

8. SEND AN EMAIL

You too can be the weird colleague who sends emails with crazy, middle-of-the-night time stamps.

9. DO SOME VIRTUAL WINDOW SHOPPING

Browse around to see what you could have purchased with the money you were just forced to spend on unplanned cloud instance provisioning in order to keep your app running. That desktop Nerf missile launcher (or 700 of them) would have been pretty nice.

10. SWEAR OFF THE GAME OF CHANCE

You’ve just proven it’s not that much fun after all. Don’t just dump everything onto one cloud and call it done. Clouds go down for so many reasons. Use an application delivery platform control layer to build in the capability to auto-switch to an available resource, while you sleep soundly. Running on multi-cloud without an abstracted control layer removes most of the value of the cloud. Swear off the game of chance. Out loud. Right now.

 

More information: