Looking Beyond Lift Testing: Apply Incrementality to Retail Media

As retail media’s popularity continues to skyrocket, finding the right metrics for measuring it remains a challenge.

The default measurement outcome across most retail media networks today is a last-touch-based attribution reported as ROAS (ad-attributed sales/spend). While there have been numerous calls to replace ROAS with iROAS (incremental return on ad spend) there is been little written on how to apply incrementality measurement to retail media in a way that is managerial useful and accounts for the unique nuances of retail media.

Marketers need to possess both a working definition of incrementality and practical approaches to measuring in a way that accounts for the nuances of retail media. 

Defining Incrementality 

The Interactive Advertising Bureau (IAB) and Media Rating Council (MRC) offer a definition in their Retail Media Measurement Guidelines that aligns with other strict definitions of incrementality; “incrementality measures the true value created by any business strategy, determined by isolating… the related results, independent of other potential business factors.”

That definition gives us a great rationale for why last-touch-based ROAS is not an effective measure - it doesn’t isolate the impact of advertising from any other potential business factors driving sales. 

This easily enables factors such as pricing, promotion, seasonality, or other advertising outside of retail media to confound results. It also highlights why attempts to reduce incrementality to a single factor like share of voice or new-to-brand (NTB) sales also fall short. They don’t account for the much broader set of factors that impact sales. 

Measuring Incrementality

While randomized control trials (RCTs) or geo-based match market tests remain an effective tool for estimating incrementality due to their ability to control for outside factors through experimental design, several barriers limit their application in retail media. 

Media buyers rarely have access to the data and tools to run RCTs within retail media networks (RMNs). The individual and order-level data needed to run these tests almost always sits behind walled gardens, making it difficult for brands to run these independently. 

Similarly, geography as a targeting and reporting attribute isn’t always available, making it difficult to leverage geo-based match market tests as an alternative. Where these two techniques can be deployed, they provide excellent point-in-time measures of incrementality or lift but two more factors complicate their applicability. 

How broadly can the results of a lift test be generalized in such a constantly changing environment as retail media? What if the conditions change relative to when the test was run, for example, a competitor's product or your product goes on promotion, or your organic rank changes? 

These factors make generalizing the results of a lift test across campaigns or even within a single campaign over time a challenge. The experiments provide an excellent estimate of lift at a point in time under a set of conditions but as those conditions change the lift results are less and less likely to be applicable. 

Practical applications

Instead, lift tests can be treated as an input to measurement in retail media instead of the output. When the results of a lift test are integrated into a broader modeling framework which is dynamics - can account for how the lift changes as other marketplace dynamics change the results a more managerial useful measure of incrementality. 

When applying things such as an incrementality measure to optimization and planning marketers can see the compelling advantage it has over last touch-based ROAS.

The danger of using ROAS in budget planning (e.g. how much do I need to invest to hit a sales target or how many sales do I think a budget of X size would yield) is that it is disconnected from the P&L due to conflicting signals from other factors. ROAS doesn’t control for outside factors that impact sales which can lead to inaccurate sales forecasts and missed budgets. 

Using ROAS for day-to-day campaign optimization is even more dangerous for this reason, leading to tactical resource allocation that doesn’t align with what is actually driving incremental sales growth for the business. 

Bringing incrementality measurement into retail will bring much-needed transparency into performance, which should be a rising tide lifting all boats. As advertising efficiency improves, sales will grow, driving ecosystem growth.