VET: Why It Matters and the Science Behind It

For the past twenty years, traditional electronic media has used frequency of exposure (i.e. impression count) as the metric to measure performance. However, times are changing and it’s become evident that this metric simply isn’t enough to measure the effectiveness of online media. This is why, at Tapad, we are proposing and offering an improved metric, Viewable Exposure Time (VET), as a more powerful and effective measure.

We know this likely begs two questions. First, what is VET? To put it simply, VET indicates the number of seconds a customer has actually engaged with media, as opposed to exposure frequency. The natural second question is: how do we know VET matters? Glad you asked, as that’s what this post is all about.

To prove the effectiveness of VET, we performed scientific research at scale to establish a premise: “VET is a fundamental aspect in analyzing media performance, above standard exposure frequency”. We performed this research using a large scale of 31 campaigns, containing 144M impressions, across multiple verticals and creatives to avoid biases.

Figure1.jpg


Figure 1 shows the results of this VET study on digital media, which were instrumented with 3rd-party viewability reporting. The y-axis displays conversion rate (CVR) which is a familiar performance metric for digital media (conversion event count/impression count). The x-axis is impression count or frequency. The three curves on the figure represent populations of users seen in the data that received low (Lower Third), average (Overall Average) and high (Upper Third) of VET.

In order to see the importance of VET, the reader should pick a particular x value (say, 15 impressions) and look at the spreading of the curves in the y direction. We see that for a given impression count, users convert at a significantly higher rate with high VET vs average and low VET. Looking at the rightmost points of the curve, we see The conversion rate for the high VET user is 60% higher than the low VET user.

In order to achieve this successful study, we had to overcome two key technical issues via data science techniques:

  • Issue #1: VET instrumentation reports only viewability in buckets, i.e. 1,5,15 seconds of viewing time.

  • Issue #2: A significant number of impressions were not measureable for viewability. However, these impressions need to be included in the VET analysis for correctness.

Here’s how we overcame these:

Solution to #1 – Interpolation of viewability event values

Per-impression reporting by the third-party partner for viewing times is bucketed with values of 1,5,15 second buckets. However, these values do not reflect actual viewing time. They indicate that the viewing time is at least as long as the reported bucket value. This distinction is important for reporting and analysis purposes.

Figure2.jpg


In order to overcome this issue, we took an approach of using an “interpolated” viewing time for each impression. The above figure shows an impression reported as in the 5sec bucket (I5) being assigned a VET value between 5sec and the next higher VET reporting bucket 15sec. This VET is drawn from a uniform distribution as shown resulting in an enriched impression I5-15, which has a non-bucketized VET.  For the analysis above, the distribution used was a continuous uniform one, though the analysis has been implemented to have this be easily changed if a more accurate distribution is determined.

The results of this interpolation?

  • Less granular set of of viewing times for impressions.

  • A more realistic situation where viewing times for a given reported value have higher moments, such as variance, etc.

Solution to #2 – Modeling of non-instrumented impressions

While all 31 campaigns used in the study were instrumented with viewability reporting, there exist situations where impressions are not measurable, for technical reasons. This results in users being served impressions such as the below illustration in temporal order:

UserEngagementx = {IR, IR, IR, IR, IR}

In the list, IR indicates an impression that was not measurable and therefore not reported upon (the 3rd and the 5th), the impressions labeled IR were impressions that were reported upon. This causes an issue for VET analysis which requires a viewing time value for all impressions. Ignoring these impressions is not a satisfactory solution, because they were delivered to the user and have an effect.

The solution executed for this issue was to impute the viewing time for the IR Impressions using probability mass functions derived from IR data. The assumption is that non-reported impressions will have a similar viewing time characteristic as the reported ones, based on some feature.

The feature we choose to base these distributions on is specific creative.  This is illustrated below for a given creative C.

Figure3.jpg


Figure 3 illustrates the modeling steps. A non-reported impression IR on creative C is modified to be a impression IR’ whose viewing time is imputed. The new impression has a viewing time value drawn from a distribution that was created using all available measurable impression data IR for creative C.

The results of this modeling?

  • 100% of impressions have VET for analysis  

Discussion

While these approaches were effective in showing the importance of VET in the presence of impression count, there are interesting aspects that can be investigated moving forward. 

  • The distribution used for the modeling of impressions was uniform; there may be a more plausible or accurate distribution. This could perhaps be investigated by looking at real distributions of viewing times, if available.

  • The feature used for the imputation distributions for non-reported impressions was creative, and this choice was based on intuition. While creative certainly makes sense, other alternatives such as placement, context, etc. could also be effective and are worth exploration.

For more information on VET, check out the below: