Optimizing for Resources and Consumers Alike

One of the genuinely difficult decisions being made by DevOps, Ops, and even straight-up developers, today is how to ensure outstanding quality of experience (QoE) for their users. Do you balance the hardware (physical, virtual, or otherwise) for optimal load? Or track  quality of service (QoS) metrics – like throughput, latency, video start time, and so forth – and use those as the primary guide?

It’s really not a great choice, which is why we’re happy to say, the right answer to the question of whether to use local or global traffic management is: both.

It hasn’t been a great choice in the past because, while synthetic and real user measurements (RUM) overlap pretty broadly, neither is the subset of the other. For instance, RUM might be telling you that users are getting great QoE from a cluster of virtual servers in Northern Virginia – but it doesn’t tell you if those servers are near their capacity limits, and could do with some help to prevent overloading. Conversely, synthetic data can tell you where the most abundant resources are to complete a computational, storage, or delivery task – but they generally can’t tell you whether the experience at the point of consumption will be one of swift execution, or of fluctuating network service that causes a video to constantly sputter and pause as the user’s client tries to buffer the next chunk.

Today, though, you can combine the best of both worlds, as Cedexis has partnered with NGINX and their NGINX + product line to produce a unique application delivery optimization solution. Think of it as a marriage of local traffic management (LTM) and global traffic management (GTM). LTM takes care of routing traffic that arrives at a (virtual or physical) location between individual resources efficiently, ensuring that resources don’t get overloaded (and of course, spins up new instances when ready); GTM takes care of working out which location gets the request in the first place. Historically, LTM has been essentially blind to user experience; and GTM has been limited to relatively basic local network data (simple “is-it-working” synthetic monitoring for the most part).

Application delivery optimization demands not just real-time knowledge of what’s happening at both ends, but real-time routing decisions that ensure the end user is getting the best experience. Combining LTM and GTM makes it simple to

  1. Improve on Round Robin or Geo-based balancing. For sure, physical proximity is a leading indicator of superior experience (all else being equal, data that has to travel shorter distances will arrive more quickly). By adding awareness of QoE at the point of consumption, however, Ops teams can ensure that geographically-bounded congestion or obstructions (say, for instance, peering between a data center and an ISP) can be avoided by re-routing traffic to a higher-performing, if more geographically distant, option. In its simplest iteration, the algorithm simply says “so long as we can get a certain level of quality, choose the closest source, but never use any source that dips below that quality floor”.
  2. Re-route around unavailable server instances. Each data center or cloud may combine a cluster of server instances, balanced by NGINX +. When one of those instances becomes unavailable, however (whether through catastrophic collapse, or simply scheduled maintenance), the LTM can let the GTM know of its reduced capacity, and start the process of routing traffic to other alternatives before any server instance become overloaded. In essence, here the LTM is telling the GTM not to get too carried away with QoE – but to check that future experiences have a good chance of mirroring those being delivered in the present.
  3. Avoid application problems. NGINX+ lets Openmix know the health of the application in a given node in real-time. So if, for instance, an application update is made to a subset of application servers, and it starts to throw an unusual number of 50errors, the GTM can start to route around that instance, and alert DevOps of an application problem. In this way, app updates can be distributed to some (but not all) locations throughout the network, then automatically de-provisioned if they turn out not be functioning as expected.

Combining the power of real user measurements, hardware health, and application health, will mean expanding the ability of every team to deliver a high QoE to every customer. At no point will user’s requests be sent to servers approaching full use; nor will they be sent to sprightly resources who can’t actually deliver QoE owing to network congestion that is beyond their control.

It also, of course, will create a new standard: once a critical mass of providers is managing its application delivery in this capacity-aware, consumer-responsive, application-tuned way, a rush will develop for those who have not yet reached this point to catch up. So take a moment now to explore how combining the LTM and GTM capabilities of NGINX+ and Cedexis might make sense for your environment – and get a step up on your competition.