Posts

Introducing the All New Sonar: a cloud-native synthetic testing tool for any infrastructure

I never guess. It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.
Sir Arthur Conan Doyle

Synthetic monitoring built for hybrid cloud
Sonar tests all of your endpoints: in your public clouds, private clouds, data centers, or CDNs. This provides a comprehensive and uniform view of the overall health of your applications delivery, no matter what the status of your various infrastructure components happens to be.
Sonar’s proactive testing acts like a virtual end user, testing to see how an application, video, or large file download would be experienced by your global customers. Being able to test your app from nine locations worldwide helps ensure your data has incredibly low latency, and therefore is actually usable for your app delivery strategy.

Ultra-low latency Synthetic monitoring, refreshed up to every other second
Public cloud users are probably used to having access to some sort of synthetic app testing functionality, as a core part of the services offered by the individual cloud provider. Where many cloud services offer services that check for availability every 30 to 120 seconds, Sonar offers checks as frequently as every two seconds. Data that’s updated every few minutes really isn’t meaningful for a solution that need to make real-time, automated delivery decisions. Not to mention the question of data objectivity when source information comes from the provider of the infrastructure being monitored.

Monitoring is passive. Cedexis is insight + action.
What makes Sonar different to other synthetic testing agents is that Sonar data can be used to shape application delivery decisions in real-time. Data collected by Sonar feeds directly into the Cedexis application delivery platform, which uses fully user-configurable algorithms to route traffic to the endpoints that deliver the highest customer experience at the lowest operational cost. Owing to the frequent health checks, and the rapid calculation of optimal traffic routes, Cedexis provides the lowest-latency cloud-based application delivery service available, with automated delivery decisions being made to route around traffic congestion less than 10 seconds after problems initially arise. By contrast, most cloud services, with less frequent synthetic checks and slower decisioning engines, may be expected to take as much as two to four times as long to respond to emerging issues.

Better data means better decisions.
Delivering applications over the internet, like all interactions with complex, dynamic systems, ultimately meets success or failure based on the data you use for making decisions. In this case, decisions are the “real-time” application delivery choices your platform makes to ensure apps and video reach your customers in a way that produces a great user experience. Using real user monitoring like Radar – the world’s largest real-time user experience community – provides data you can use to make automated delivery decisions on your hybrid infrastructure. But to enable your application delivery logic to fully understand and optimize delivery for all of your customers and potential customers worldwide, you need to proactively test networks. That’s where Cedexis’ Sonar functionality comes in.

The three pillars of Application Delivery
Cedexis application delivery platform is powered by three powerful services:

  • Radar: the world’s largest community of instantaneous and actionable user experience data
  • Fusion: a powerful 3rd party data ingestion tool that makes APM, Local Load Balancer, cloud metrics, and any other dataset actionable in delivery logic
  • [NEW!]: Sonar: a massively scalable and architecture-agnostic synthetic testing tool that is immune to the latency issues of proprietary cloud tools

 

The Cedexis application delivery platform automates and optimizes the customer experience for apps, video, and static content while minimizing cloud and content delivery costs. This is done by combining billions of real user data points from over 50,000 networks, Sonar synthetic testing data, and any other dataset you use to optimize delivery based on real user data from our entire network (not just your customers).
If you haven’t created a Cedexis portal account yet, now’s the time. You can set up your global application delivery in a few minutes and see how Sonar works for yourself.   

Cedexis Solves Avoidable Outages in Real-Time

Portland, Ore. – August 15, 2017 Cedexis, the leader in crowd-optimized application and content delivery for clouds, CDNs and data centers, today announced the release of its connected Sonar service, which uses low-latency synthetic monitoring to eliminate costly and avoidable outages by ensuring consistent application delivery. Providing exceptional quality of experience (QoE) to application consumers by eliminating outages and slowdowns is at the heart of building a profitable and sustainable cloud-native service, but has proven to be an elusive goal.

Synthetic monitoring uses programmed requests of customer-designated endpoints to validate, on an ongoing basis, that those endpoints are available for use. Until now, the marketplace has offered only two substantial choices:

  • Implement disconnected synthetic monitoring, which requires manual intervention when problems arise. The time from anomaly detection to resolution can range from minutes to hours, often resulting in prolonged outages and slowdowns.
  • Implement cloud vendor-specific synthetic monitoring, which delivers automatic intervention when problems arise. The time from anomaly detection to resolution is measured in just minutes, but is generally restricted to re-routing within that vendor’s cloud infrastructure, resulting in shorter, but still meaningful, outages and slowdowns.

With the release of Sonar, there is now a third, and more effective, option:

  • Implement Sonar connected synthetic monitoring, which delivers an automatic intervention when problems arise. The time from anomaly detection to resolution is measured in just seconds, and traffic can be re-routed across and between substantially any infrastructure (from data center to hosting facility to cloud provider), resulting in the elimination of most outages and slowdowns entirely.

Sonar is able to reduce the MTTR (mean time to repair) – and thus prevent consumer-visible outages and slowdowns – automatically owing to two key characteristics:

  1. Sonar is connected to the broader Cedexis application delivery platform. As such, the data that is collected automatically flows into the Openmix global traffic manager, which is able to adjust its traffic routing decisions in just seconds.
  2. Sonar is configurable to run endpoint tests as frequently as every two seconds, providing up-to-date telemetry moving at the speed of the Internet. By contrast, cloud vendor-specific solutions often limit their testing frequency to 30 – 120 second intervals – far from sufficient to contend with rapidly-evolving global network conditions.

“Synthetic monitoring data, as a core input to traffic routing decisions, must be accurate, frequent, and rapidly integrated into algorithms,” said Josh Gray, Chief Architect at Cedexis. “However, data is only as valuable as the actions that it can automatically activate. Updating traffic routing in just seconds is the key to making outages a thing of the past and ensuring unparalleled user experience.”

The updated Sonar synthetic monitoring service enhances the industry-leading actionable intelligence that already powers the Openmix global traffic management engine. The Cedexis application delivery platform (ADP) uniquely uses three different sources of actionable data to ensure the smoothest internet traffic logistics:

  • Radar: the world’s largest community of instantaneous user experience data
  • Fusion: the powerful 3rd party data ingestion tool that makes APM, Local Load Balancer, Cloud metrics, and any other dataset actionable in delivery logic
  • Sonar: a massively scalable and architecture-agnostic synthetic testing tool that is immune to the latency issues of proprietary cloud tools

“No global traffic management platform can provide reliable, real-time traffic shaping decisions without access to accurate, actionable data,” noted Ryan Windham, Cedexis CEO. “The evolution of Sonar to provide industry-leading latency levels confirms our commitment to delivering an end to avoidable outages.”

RUM vs Synthetic – why people matter

LightBulb-in-the-lead

“Cedexis Leads the Pack”.

It is nice to hear – especially when it comes from a prestigious analyst firm. If you have not seen this report, its worth the read. Cedexis was recognized for innovations in the monitoring space. While we are clearly honored, its somewhat ironic because we give our Real User Monitoring (RUM) away for FREE to all Radar community members. What this report clearly shows is why RUM is significantly better than synthetic monitoring for certain kinds of things. This is not to say that Synthetic monitoring does not have a place – but for real time traffic routing RUM is the best solution. Let me give you an example of why this is true.

As an experiment – lets take 6 global CDNs and point synthetic monitoring agents at them. The 6 CDNs are Akamai, Limelight, Level3, Edgecast, ChinaCache and Bitgravity. I am not going to list their results by name as we are not trying to call anyone out. Rather I mention them here just so the reader knows we are talking about true global CDNs. I am also not going to mention the synthetic monitoring company by name – but suffice it to say they are a major player in the space.

We point 88 agents, located all over the world, to the small test object on these 6 CDNs we benchmark. Now we can compare the synthetic agent’s measurements to the Cedexis Radar measurements for the same network from the same country, each downloading the same object. The only difference is volume of measurements and the location of the agent. The synthetic agent measures about every 5 minutes whereas Radar measurements can exceed 100 measurements per second from a single AS. Of course, the synthetic agents are sitting in big data centers versus Radar running on real user’s browsers.

One more point on the methodology: since we are focused on HTTP Response; we decided to take out DNS resolution time and TCP setup time and focusing on pure wire time. That is First Byte + Connect time. DNS resolution and TCP Setup time happen once for each domain or TCP stream whereas response time is going to impact every object on the page.

We will look at a single network in the US. The network is ASN 701: “UUNET – MCI Communications Services Inc. d/b/a Verizon Business” (USA). This is a backbone network and captures major metropolitan areas all over the US. Cedexis Radar received billions of measurements from browsers sitting on this network within the US.

Screen-Shot-2015-04-08-at-12.59.28-PM

Clearly, CDNs are much faster inside a big data center then they are in our homes! More interestingly are the changes in Rank; Notice how CDN1 moves from #5 to #1 under RUM! Also the scale changes dramatically, the synthetic agents data would have you believe CDN6 is nearly 6X slower than the fastest CDNs – yet when measured from the last mile they are only about 20% slower.

So if you had these 6 CDNs in your multCDN federation and were doing Latency Based load balancing based on these synthetic measurements – the people on this network would be poorly served. CDN1 would be getting very little (if any) of the traffic from this network even though its the fastest actual network. RUM matters because thats where the people are! By measuring from the datacenter you obfuscate this important point.

Synthetic agents can do many wonderful things but measuring actual Web Performance (from actual real People) is not among them; performance isn’t about being the fastest on a specific backbone network from a datacenter, it is about being fastest on the networks which provide service to the subscribers of your service. The actual people.

RUM based monitoring provides a much truer view of the actual performance of a web property than does synthetic, agent based monitoring. We urge you to go deploy our Radar tag and see for yourself who is performing best right now. Our real-time RUM measurements provide the best possible view into how global CDNs compare with each other in every region of the world.

Monitoring: What RUM are you getting served?

Real user monitoring, or RUM, is a method for collecting metrics from end-users of a website or web-based application using some kind of passive monitoring software. The quality of a digital experience, including response time, reflects powerfully on brands. Understandably, the measurements and the insights RUM analysis can provide are a hot topic in the content provider industry. But do you know what brand of RUM you are really getting?

The most advanced website monitoring tools right now usually use either Real Users or a synthetic analog. It’s important to understand the difference. Synthetic testing runs or emulates a web browser. This software can be quite sophisticated and usually does a good job of acting exactly like the application it’s imitating.

Using either custom-built scripts or an analysis engine, the synthetic testing software generates multitudes of clicks and events on the website. Thousands of measurements can be quickly gathered this way, and analysis of the resulting data set can yield a great deal of useful information. However, though this method can efficiently collect excellent results in a short time, it has two very important deficiencies compared to our favorite RUM, Cedexis Radar Real User Measurements: First, it is a simulation of users, Second, it does not account for the critical last mile of the network.

Simulated Users and Confirmation Bias

Synthetic users are not real users. Manually scripted synthetic sessions are subject to that subtle confirmation bias that plagues test engineers by subconsciously discouraging them from triggering those odd outlying bugs actual users always seem to find. Classic problems with confirmation bias will hamper the site: Edge cases are not captured, real end-user practices are overlooked, and results can not be correlated to actual end-user experience.

The Vital Last Leg

Synthetic user monitoring lacks that important last leg of network traffic essential to measuring the response time experienced by a real end-user. Many synthetic environments exist in major Internet hubs or within firewalls, and thus do not suffer the problems with latency and connectivity that many end-users deal with. The high-speed, low-latency connections on which synthetic testing platforms live can therefore miss certain important speed issues, such as the impact of narrow band mobile connections, or congested ISP peering, inducing rendering delays caused by big and improperly constructed HTML. This diminishes the utility of synthetic testing in certain areas where real user experiences are important.

Before the release of an application, synthetic testing allows evaluation throughout development, and it is often the most effective way to stress test. Likewise, Cedexis Radar RUM requires the collection of a user base before it is effective, and it would be irresponsible to wait until this is established before performing real-time testing of an application.

Apply the Right Tool

The primary difference between Cedexis Radar RUM and synthetic testing is that RUM is crowdsourced data collected from real users, and synthetic testing is performed by software agents attempting to emulate real users.

While synthetic testing can be launched from dozens or even hundreds of locations, it cannot effectively simulate the true diversity of RUM, which utilizes measurements from potentially millions of end-points in every country and tens of thousands of ISPs. But the other differences point out shortcomings in the metrics that synthetic testing gathers and, for that reason, it’s important to understand the difference between the two.

In my next post, we will explore the best ways to use these tools in cloud or content delivery environments.