ISP issues in video delivery are one of the biggest pain points for operations teams, which is saying something because delivering streaming video itself is already difficult and not as straightforward as it seems.
Although it follows a basic path of origin to CDN cache to end-viewer through the ISP, there are a lot of nuances to how that path is architected. This creates challenges for ensuring a high end-to-end Quality of Service (QoS).
The main one is getting the right data quickly enough to make real-time business decisions. That’s because while there are many vendors within the workflow that expose their data programmatically, there are still blind spots: places within the flow of content that are critical to QoS but hard for the streaming operator to obtain data from. The chief culprit is the connection between CDN and ISP.
Understanding the main CDN, player, and ISP issues in video delivery
In today’s streaming video architectures, the big commercial CDNs are often cross-connected with ISPs in the major peering fabrics. Although this facilitates better delivery, by eliminating a lot of travel across the open internet, it doesn’t result in better operational visibility. Because when there are issues between the two networks, the CDN and the ISP, streaming operations engineers need to put on their detective hats and comb through player analytics and CDN logs to try and understand the problem.
Moreover, even though they can eventually trace a player issue to a specific CDN, they can’t see the ISP network in between, which is a blind spot in the data flow from CDN to player. Perhaps the request is malformed or corrupted within the ISP network, or there is a problem within the peering point that results in the CDN not receiving the request. There could also be an ISP network error, so the request never reaches the CDN. Regardless of the exact ISP issue in video delivery, understanding the connection between the player making the request, the CDN receiving it, and the ISP through which request and response are handled, is critical to meeting the viewer expectations for QoS.
Real-time is part of the solution (and the problem) in ISP issues in video delivery
In the last two years, most commercial CDNs have begun supporting real-time log delivery. For example, both Amazon and Akamai have made announcements regarding their support for real-time logs, which enable streaming operators to see what’s happening between player and CDN and make informed decisions quickly. However, that same influx of millions of lines of log data from multiple CDNs, combined with upstream data (such as encoders) and downstream analytics (from players), is often overwhelming and time-consuming to analyse.
Additionally, although all that data can provide insights into addressing QoE issues, it doesn’t make up for the lack of data about the ISP. Fixing that blind spot requires a different approach than brute forcing data into a dashboard.
Best way to overcome ISP issues in video delivery: going from raw data to summaries to the perfect dashboard
The key to solving the blind spot is to develop a summary of those real-time CDN logs and the player analytics to create a visualisation of the flow of data.
Yes, most streaming operators employ powerful data tools like Splunk, which provide great dashboards. Yet, those tools don’t summarise the data in any meaningful way. It’s just the raw data that doesn’t facilitate observability. A summary, especially when it’s visually marked by colour, helps a streaming operations engineer identify the root cause much faster. They start with the visual indicator of a problem, then drill into the data to see where the issue is rather than spending time trying to figure out where to click.
That is precisely the key to efficiently overcoming ISP issues in video delivery. Using a summary-based approach, the scope of the data is visualised against calculated KPIs: the health of the delivery pipeline between each CDN edge node and Autonomous System Number (ASN) is colour-coded. This empowers an operations engineer to drill into a connection for more KPI details, which eventually leads to an understanding.
However, how can you get such a summary? For Touchstream’s ISP Insights, it’s through a combination of Lambda functions and a shared S3 bucket. Touchstream picks up the data with a Lambda function and then utilises another to summarise the data into a usable pool, against which the ISP Insights dashboard is able to visualise that CDN-to-ASN relationship. The result is reduced MTTR and MTTD, which translates to happier viewers and less churn
👉 You may like: The Complete Guide to Live Stream Monitoring
Keeping security, privacy, and cost in mind when tackling ISP issues in video delivery
Of course, streaming operators using a third-party tool like ISP Insights with such large datasets have to keep cost in mind. It’s not cost-effective to have the dashboard vendor re-ingest the data. This is why the Lambda functions play such a critical role: they pull a subset of the data. Subsequent functions within the dashboard environment calculate the KPIs. This keeps egress and ingress costs down while maximising the value of the data.
Moreover, this approach has another benefit as well: security and privacy. Log data often contains personally identifiable information (PII). By keeping the source data secure in the operator’s datastore and using the Lambda functions to pull only what’s needed (essentially scrubbing the data of any PII), the operator is assured that privacy and security are maintained while solving the blind spot ISP issue in video delivery.
With Touchstream ISP Insights, ISPs will never be blind about last-mile video delivery issues
Solutions like those offered by Touchstream enable you to gain end-to-end insight into your workflow; to visualise it with colours and provide easy drill-downs for deeper understanding. However, even with that, there will always be a blind spot. Without the telemetry between the CDN edge node and ASN, the gap in that visualisation will continue to hamper an operator’s ability to provide the highest level of QoE in real-time.
With Touchstream’s ISP Insights, it’s now possible to fill in that gap within the end-to-end workflow monitoring dashboard so operations engineers can have a truly holistic view of the video experience. Touchstream’s ISP insights, together with summary-based approaches and real-time log delivery, empower operators to efficiently overcome ISP issues in video delivery and ensure a high end-to-end QoS for their viewers.
If you’ll be attending NAB and want to see a demo of Touchstream ISP Insights, schedule a meeting here.
