Video Streaming Metrics

To provide an accurate description of the overall quality of experience delivered to your users, Codavel tracks the most relevant video performance, and engagement metrics for each video playback your users attempt during a session.

A user session is defined as the period of time a user starts interacting with your app until closing it or sending it to the background, and includes all the playback attempts a user performs during that time.

You can find the list of metrics collected below:

Engagement Metrics

• Unique Users

• Videos Played

• Watch Time

• Total Waiting Time

• Exits Before Video Start

• Drop-offs

Video Performance Metrics

• Playbacks

• Startup Time

• Rebuffering

• Video Quality

Note:

On your dashboard, you will find aggregated values of your video performance and engagement metrics that you can drill down to understand the state of individual sessions.

General - Player Time Definition

Codavel Analytics module defines PlayerTime (ms) as the partial or total sum of the following performance metrics:

• Startup Time: Measures the startup time of a video playback in milliseconds (ms) - from when the user presses play until the 1st frame is available for playing.

• Watch Time: Corresponds to the time the video player is actually displaying the video, excluding time spent on startup, pauses, buffering, and seeking throughout the video.

• Buffering Time: Time in milliseconds (ms) the video player stops playing due to unintended buffering. This does not include startup time, paused time, or time spent buffering due to seeking throughout the video.

• Seeking Time: Time in milliseconds (ms) the player spends on buffering specifically due to a video seeking action from the user (the user went forward or backward in a video, causing a buffering event).

Codavel tracks each video experience until the respective playback has ended (e.g. the user dropped off the video at a certain point or the video finishes). Keep in mind that, if a user presses play, the video loads but the user leaves before it starts, that counts as a video played (startup time was recorded). If a user presses play, watches part of the video, pauses and then resumes, that will also count as a single video played. The same happens if a user seeks through the video without leaving the ongoing playback - even if the user is watching the video in loop, that will count as a single video played.

Description:

In this example, the user clicks on a video (triggering the beginning of the startup time):

• (@0s) Startup time begins;

• (@2s) The 1st video frame is available to be played, hence ${StartupTime = 2s}$.

• The video then plays for 8s, meaning so far ${WatchTime = 8s}$.

• (@10s) The video stops due to a buffering event that lasts 2s (note here that the video only buffers because of network conditions - the user did not intend for its video to be interrupted). This means that ${BufferingTime = 2s}$.

• (@12s) After this buffering event, the video resumes playing for another 8s, meaning ${WatchTime = 8s}$ (previous watch time) ${ + 8s = 16s}$.

• (@20s) The user seeks forward into the video (note that video seeking can be both forwards or backwards), causing the video to buffer again for 2s. Since this buffering event is caused by a seeking action from the user, it causes the ${SeekingTime = 2s}$.

• (@22s) After seeking, the user watches 3s of video (meaning so far ${WatchTime = 19s}$.) and leaves at the @25s mark.

Calculation:

PlayerTime of this example is equal to:

${ PlayerTime = ( StartupTime + WatchTime + BufferingTime + SeekingTime )}$

${ PlayerTime = 2s + 19s + 2s + 2s = 25s }$

Engagement Metrics

Your users' behavior and engagement are highly dependent on the quality of the experience delivered. It is expected that a video playing with no friction sees more engagement (e.g., more time watched, fewer drop-offs) than a video that keeps stalling or playing in low quality. To understand what type of experience is being delivered to your users, we resort to the following engagement metrics:

Unique Users

To identify the number of unique users, we count the number of distinct users (via anonymized unique IDs) that started interacting with your app during a selected time period.

Calculation:

${Count}$ (users with a unique ID)

Videos Played

Every time a user presses the “play” button or the video starts programmatically (playback time was recorded) we count and increment the number of videos played.

Calculation:

${Count}$ (number of videos played)

Watch Time

Watch Time corresponds to the time the video player is actually displaying the video, excluding time spent on startup, pauses, buffering, and seeking throughout the video.

WatchTime

Measures the watch time of video playback in milliseconds. It is the time the video player is actually displaying the video, excluding startup time, paused time, buffering time, and time spent on seeking throughout the video.

Calculation:

${Count}$ (ms spent playing video)

WatchPercentage

Percentage of time the video player spent on playing the actual video, in relation to PlayerTime.

Calculation:

${( \frac{Watch Time}{Player Time} ) * 100}$

Total Waiting Time

Time the user spent waiting until drop off occurred.

Calculation:

${Player Time - Watch Time }$

Exits Before Video Start

An exit before video start happens when a user abandons a video before it starts playing, usually because it's taking too long to load. We count this event when the user presses the play button on a video or scrolls/swipes to a video that begins to play programatically.

Calculation:

${Count}$ (number of "play" clicks with no video displayed)

Drop-offs

A drop-off occurs whenever a user exits a video prematurely (e.g., the user “gives up” on this video, not watching it until the end). When this happens, we count it as a drop-off and record the state of the player conditions at the moment, providing information about where a user was in the journey when the drop-off occurred.

Calculation:

${Count}$ (drop-offs before video completion)

These metrics are available in the Codavel Dashboard (Video KPIs Tab), in your Private Area (opens new window).

Video Performance Metrics

Tracking video performance is crucial to understand your overall quality of experience. More often than not, video performance heavily impairs user engagement. No matter how good the video content may be, if it’s not performing as intended (e.g., fast start, no stalling, and good quality), you will probably still be seeing a lot of people abandoning videos prematurely or leaving the app after a short session. To understand if this is happening because of poor video performance, we resort to the following (and most relevant) video streaming metrics:

Playbacks

Every time a user wants to start watching a video (by pressing the play button or scrolling/swiping to a new video that starts programmatically), we count it as a Playback attempt.

During this Playback attempt, the video playback can be subject to 2 types of Playback Failures: Fatal and Non-fatal Playback Failures. A Fatal Playback Failure prevents the video from being played entirely, while a Non-fatal Playback Failure can still allow for the video to be played.

Regarding Playbacks, consider the following metrics:

Fatal Error Count

Count of the total number of fatal errors. A Fatal Error is an error that prevents the video from being played.

Calculation:

${Count}$ ( Fatal Errors )

Non Fatal Error Count

Count of the total number of non-fatal errors. A Non-Fatal Error is an error that does not prevent the video from being played.

Calculation:

${Count}$ ( Non-fatal Errors )

Consider these metrics to:

• Understand the impact of playback failures on your overall user experience.

• Find out when playback failures are happening the most and how frequently.

Startup Time

Time interval from when the user selected a video (play button was clicked or the video started in auto-play) until the 1st video frame is available for playing.

Two metrics are collected based on this interval:

Startup Time

Measures the startup time of a video playback in ms - from when the user selected a video until displaying the 1st video frame.

Calculation:

Time between pressing the play button (player is ready) and displaying the 1st frame of the video.

Startup Percentage

Percentage of time the video player spent on startup in relation to PlayerTime.

Calculation:

${( \frac{Startup Time}{Player Time} ) * 100}$

Consider these metrics to:

• Understand the impact startup time has on the number of exits before video starts and user engagement

• Understand when startup time should be optimized.

• Find out performance discrepancies in startup time for your average user base and tail-end users.

Rebuffering

Time the video player stops playing due to buffering out of just network conditions. This does not include startup time, paused time, or time spent buffering due to seeking throughout the video.

Three metrics are collected based on this value:

Rebuffer Count

Count of the total number of times a video playback entered the buffering state due to network conditions.

Calculation:

${Count}$ ( Total Rebuffer Events )

Rebuffer Duration

Sum of total time spent in the buffering state in ms, due to Network issues.

Calculation:

${Count}$ ( Total Rebuffer Time)

Rebuffer Percentage

Percentage of time the video player spent on the buffering state, due to network conditions in relation to PlayerTime.

Calculation:

${( \frac{Rebuffer Duration}{Player Time} ) * 100}$

Consider these metrics to:

• Understand the impact rebuffering has on user engagement (e.g., watch time, premature drop-offs, number of video completions).

• Find how frequently your users are interrupted by rebuffering events.

• Find how long your users are waiting for their content, per rebuffering event.

• Rethink adaptive algorithms or rebuffering strategies.

Video Quality (Bitrate)

Bitrate is the network capacity for delivering. It is an indicator of the video quality the users are experiencing during a video stream.

Two metrics are considered:

Initial Video Bitrate

Measure of the first video bitrate seen in a video playback, in kbps.

Mean Video Bitrate

Measures the average video bitrate seen in a video playback in kbps. It is computed as the weighted average of the bitrates of video playback, taking into account the duration of the requested bitrate.

Calculation:

Weighted average of the amount of time spent at each bitrate while a video is played.

For example: if during a view lasting 3 minutes a video plays for 1 minute at 1Mbps and 2 minutes at 2Mpbs, the Weighted Average Bitrate would be:

${Mean Video Bitrate = \frac{(1Mbps \times 1min) + (2Mbps \times 2min)}{3min} = 1.67Mps}$

Consider these metrics to:

• Understand how video quality affects your user engagement (e.g., watch time, premature drop-offs, number of video completions).

• Understand when bitrate should be optimized.

These metrics are available in the Codavel Dashboard (Video KPIs Tab), in your Private Area (opens new window).