Streaming operators struggle to provide exceptional viewer Quality of Experience (QoE). Not because they aren’t skilled, but simply because the video streaming tech stack is a hodge-podge of different technologies, sometimes hosted by third-party providers, with little to no interoperability. This complicates identifying the upstream or downstream impact of an issue as traffic moves through the streaming workflow.
Thankfully, a new project from the Consumer Technology Association’s (CTA) Web Application Video Ecosystem (WAVE) group and the Streaming Video Alliance (SVA) helps to solve this problem. By providing a standardised approach to monitoring across the entire stack, this new Distributed Request Tracing project will allow operators to trace issues down to individual user sessions and unify their end-to-end monitoring while improving overall observability.
Tackling the challenges of end-to-end streaming workflow monitoring
As the video streaming tech stack has evolved with microservices, cloud streaming, and serverless functions, it has become harder to generate observability without a significant amount of engineering effort. Thus, streaming operators face two main challenges today: telemetry without interoperability and tool fragmentation.
The former requires a lot of post-processing and engineering work to fashion an end-to-end view using data that is not standardised. That lack of standardisation makes it very difficult to stitch together the trail of data through the workflow. Regarding the latter, in some cases, the components within the streaming video tech stack are cloud services provided by SaaS operators who have their own monitoring tools. This requires streaming operators to switch between tools to get an end-to-end view of their video workflow. So, a lack of standardisation of streaming data, combined with a variety of different tools, leads to increased time to determine root cause analysis and fix problems before they result in viewer abandonment or subscriber churn.
Distributed Request Tracing: A New Hope For Streaming Workflow Monitoring
The joint CTA-WAVE and SVA Distributed Request Tracing project attempts to address the challenges of non-standardised streaming workflow monitoring and tool fragmentation. It proposes to link together the streaming video stack data through session ID and request ID arguments (leveraging other standards, like CMCD) to enable streaming operators to more easily track issues through the OTT workflow on a per-session basis and create “baselines” of component operations that can be used to monitor groups of components more holistically. In turn, this will generate better observability.
Distributed Request Tracing does this by standardising session IDs, and request ID arguments are represented within different data sets and handled by different technologies. This enables streaming operators to link data together to provide a more holistic view of the components. Think about it this way: how do you know a specific segment, output by an encoder and delivered by a CDN, is linked to a playback issue when all of those technologies represent session IDs differently and expect different request ID arguments? Some of the standardisation can be handled by other efforts, like CMCD, but the only way to link everything together across components (and not just the player) is through this new project.
Moreover, Distributed Request Tracing isn’t just about identifying issues and the relationship between components within issue resolution. It’s also about improving observability. When streaming operators don’t have to spend time post-processing and figuring out how to connect datasets themselves, they can utilise powerful tools to surface potential problems and other insights, which might otherwise be unavailable across disparate, unconnected datasets.
Of course, a standard like what the CTA-WAVE group and the SVA are working on is just the beginning. Streaming operators will need a tool to help them aggregate, segment, and ultimately visualise the data from across the streaming video workflow. That tool is an end-to-end dashboard which can provide several value benefits built on a standard interoperability between streaming video workflow datasets:
- Visualisation: Operations will be able to see all of the datasets of the workflow represented visually. This can be colour-coded to identify data within the technology that is in variance from expected norms
- Drill-down: Through the visualisations, operations can easily drill into specific data sets to see which individual sessions are experiencing problems. Because all the datasets are connected via a standardised session identifier, operations can easily trace the problem to its root, such as an ISP or CDN outage
- Observability: With everything being connected and colour-coded end-to-end in a single visualisation, operations can see relationships between data sources that might not have normally been considered and utilise the drill-downs and other data details to see how sessions (in aggregate) are affected
How operators can improve streaming workflow monitoring while preparing for the standard
The distributed request tracing effort, which began in the SVA and was then pulled into CTA-WAVE for standardisation, is still in its beginning stages. As such, it may be a few years before technology vendors in the video streaming tech stack implement the standard so that it’s available for streaming operators in their end-to-end dashboard.
So what can streaming operators do in the meantime to improve observability across their streaming workflow and provide a consistent QoE? One of the immediate ways is to build consistency across your CDN logs. Although this may take some initial time to create a data model that post-processes the logs from your different CDNs into a consistent format within your dashboard, it will be worth it. Furthermore, you can even use CMCD to tie the logs together for tracing purposes. By focusing on the CDN logs now, though, it will also help reduce tool fragmentation–operations will no longer need to log into the CDN to use their specific streaming workflow monitoring tool.
You don’t have to wait to start end-to-end streaming workflow monitoring
Whether you are monitoring live streams or a lot of on-demand traffic, you can’t wait for the standard to improve your end-to-end view. With so many streaming options available to consumers today, QoE is rapidly becoming a key differentiator along with original content. Of course, the CTA-WAVE/SVA distributed request tracing project will resolve some significant streaming workflow monitoring challenges. However, it will still require an end-to-end dashboard, which is something that you can start implementing today on top of a monitoring harness to collect and visualise all of that data.
Thankfully, Touchstream, who is a key contributor to the distributed request tracing work, has done the heavy lifting for you. By implementing their end-to-end monitoring solution, you’ll be ready to visualise the standardised data and take your monitoring to the next level.
Interested in better streaming workflow monitoring? Schedule a demo today.
