How to Use Lift Tests to Evaluate Marketing Campaign Impact?

Are you having a tough time figuring out why conversion or engagement rates are declining? You're not alone. Customer journeys have been getting incredibly complex lately. Brands spend more to understand their customers better but are hobbled by poor attribution and scattered data sources.

Lift testing isn't new, but marketers are increasingly using it as a precision tool for testing assumptions and optimizing marketing strategy.

What makes it so effective? For one, it directly measures the increase in desired outcomes (like sales or conversions) attributable to a specific marketing initiative. This contrasts with other methods that may infer effectiveness indirectly.

In this article, we'll discuss ways you can leverage lift testing to get the most out of your marketing efforts.

What is a Lift Test?

A lift test, also known as incrementality testing, is a method used in marketing to measure the additional impact or "lift" that a specific marketing campaign or strategy has on a key performance indicator (KPI) such as sales, conversions, or engagement, compared to a baseline where that campaign was not executed.

Lift tests work by randomly dividing an audience into a control group and one or more treatment groups. The control group does not receive an offer at all, while the treatment group does. The performance of the treatment groups is compared against that of the control group. The difference in key metrics between the two is the incremental lift.

For example, if 5% of customers who didn't see an ad purchased 10% of those who did, then the ad campaign is said to have a two-fold lift. So a statistically significant result here would mean there's a low probability that the observed (increase in sales/improvement in brand awareness) is due to random chance. This suggests the ad itself likely played a role.

You can lift test everything from channels and messaging to customer segments to find out what's working and what's not. Depending on the goal, you can use metrics like conversion rate, customer acquisition cost (CAC), average order value (AOV), and so on to measure lift.

Why Lift Testing is a Game Changer for Marketers?

Lift testing is a game changer for marketers because it directly links specific outcomes to their causes, going beyond the constraints of traditional methods that primarily track general trends. This method provides precise attribution of sales and robustly justifies marketing decisions with solid evidence, thereby enhancing the accuracy and accountability of marketing initiatives.

In contrast to correlation-based techniques that presume a linear progression from the first click to conversion, lift testing recognizes the complex, non-linear nature of omnichannel customer journeys. It pinpoints incremental impacts throughout the customer funnel and provides a clearer understanding of causality in customer behavior.

Lift testing offers detailed insights into the effects of marketing interventions on outcomes like clicks, sales, and sign-ups. It refines targeting at crucial touchpoints, consistently improving engagement and return on investment. Furthermore, by identifying the root causes of customer behaviors, lift testing enables marketers to proactively adjust strategies to meet desired goals. This process helps secure management support for expanding successful campaigns to broader audiences.

Preparing for Lift Testing

Here's a checklist to follow when preparing for your lift test:

1) Define the goal you want to achieve: This is where you define the goal in clear, specific terms. Describe the specific marketing intervention you want to test, your target audience, and the outcome you're hoping for.

2) Set clear objectives and KPIs: Having defined the goal, you can now drill down to the objectives and KPIs for the test. In this case, your primary objectives could be to assess the effectiveness of a promotional discount on sales, identify whether repeat customers or first-time buyers drive revenue and assess incremental lift. Accordingly, you can pick conversion rate, total sales, or ROI as the KPIs to focus on.

3) Define the variables you want to test: Let's say you want to run Facebook ads, as that's where most of your sales come from. So, Facebook ads would be the intervention variable, with the conversion rate being the outcome variable. Changing either the platform or the variable would alter the outcome.

4) Identify sample size and timelines: It's crucial to have a large sample size for testing. But how do you determine the right size to begin with? A rule of thumb is to include at least 100 samples per group. The exact sample size can vary depending on factors like variability and desired statistical power. Secondly, the test duration should be long enough to allow for the collection of incremental data. The longer, the better.

Designing Your Lift Test

Here's how to structure your lift test after setting clear objectives and selecting KPIs and variables:

1) Assign users to control and treatment groups: Properly segment users into control and treatment groups ensuring no overlap exists between the groups. It's essential to maintain homogeneity in characteristics such as demographics and purchase history within these groups. Use automated tools like Lifesight for precise segmentation based on custom criteria.

2) Segment your audience: Effectively segment your audience based on demographics, behavior patterns (such as page views, sign-ups, downloads), interests, and needs to ensure balanced group formation.

3) Establish statistical significance and confidence intervals: Determine the statistical significance to understand the reliability of the lift effect across the audience. Confidence intervals will indicate the range within which the lift is likely to occur, providing insights into the effectiveness of the intervention.

4) Test one variable only: To maintain clarity in the results, limit the test to one variable at a time. This simplifies analysis and enhances the validity of the results. For more comprehensive insights where multiple variables need examination, consider multivariate testing.

5) Check data quality: Ensure data integrity by auditing and correcting any discrepancies. Verify that all data collection tools, such as pixel tracking and sign-up forms, are functioning correctly to prevent data loss.

6) Create ad variations: Develop distinct ad creatives for different offers. Ensure each variation communicates the value proposition and includes an appealing call to action. Additionally, manage your ad budget and scheduling to maximize exposure during the campaign.

Implementing the Lift Test

1) Decide what type of lift test to use: Two of the most common types of lift tests are geo-experiments and audience split tests. Geo-experiments are ideal if you want to compare sales performance across segments by region.

For example, you can segment customers based on the east and west coast. On the other hand, audience split tests are better suited for testing customer responses to specific message variations and creative formats. You can use a combination of both for deeper insight.

2) Pick the right marketing measurement tool: Using the right automated tool for lift testing can take a huge chunk of work off your plate. You can use free tools like Google Optimize or Facebook Conversion Lift (excluding ad account charges). However, these are platform-specific.

For omnichannel campaigns, consider an integrated marketing measurement tool like Lifesight. In addition to running lift tests at scale, it can simulate various counterfactual scenarios for deeper insights into ROI.

3) Set implementation SOPs: Put in place Standard Operating Procedures (SOPs) for your team members to follow during the test. Let all stakeholders know the testing schedule and goals. Assign responsibility for monitoring the test and responding to technical or operational problems.

4) Run the experiment: Upload your audience data to Facebook and start your campaign. For the campaign's duration, keep track of vital metrics like conversion rate and total sales. Let's say you run the campaign for 30 days. Finally, collect data from both groups (control and treatment) and prepare it for analysis.

Analyzing Lift Test Results

1) Measure lift: Based on the data, let's say you find that the baseline conversion rate (control group) was 4%. In comparison, that of Group A (existing customers) was 10% (150% lift), and Group B (new customers) had a 3.5% conversion rate. To measure the true impact, you need to use statistical techniques like Difference in Differences (DID), Regression Analysis, and Chi-Square based on factors like audience size, data range, etc.

These tests will put the observed lift into perspective and reveal its contribution. They will also reveal the degree to which sales will likely change (variability) if you continue offering the 20% discount over time.

2) Check if the lift is statistically significant: Imagine the lift score was a point on a horizontal line representing the confidence interval, with a midpoint representing "no change." The farther away it falls from the midpoint, the more statistically significant lift is considered. Conversely, if the lift estimate is placed too close to the midpoint, the inference is that the lift is weak or non-existent.

The level of statistical significance indicates that the discount offer drove higher sales with existing customers. There are many nuances to measuring statistical significance, especially with large sample sizes and multiple variables to be tested. An automated tool like Lifesight can make it much easier for you to scale and optimize lift testing.

3) Put the result into context: Next, apply the findings to your original hypothesis to understand the real-world implications. For example, if the lift is statistically significant, our 20% discount positively impacted sales.(Consider expanding the discount offer to a wider audience if the lift is large enough. If it's minimal, it may not be worth the cost.)

Optimizing Campaigns Based on Lift Test Insights

Lift testing significantly enhances the efficiency of marketing spending through an iterative approach. By identifying the causal effects of the best-performing campaigns, lift testing enables marketers to refine successful strategies, discard ineffective ones, and deliver targeted messages to customers at different stages of their buyer journey. This method improves personalization and the prediction of customer behavior across segments, allowing for the optimization of touchpoints like checkout pages and sign-up forms to address issues such as cart abandonment or low sign-up rates.

You can use the insights from the lift test to tweak existing campaigns and inform future strategies. More specifically, you can holistically align channels, formats, and messaging to drive omnichannel engagement and reduce Customer Acquisition Costs (CAC).

That said, integrating data from across the funnel without loss, validating it, and running lift tests is time-consuming. If your data collection process has gaps, testing results will not be reliable, and scaling will be complex.

Once you've measured lift, tools like Lifesight let you calibrate the results with other models like Multi-Touch Attribution (MTA) and Marketing Mix Modelling (MMM) and solidify your strategy.

Challenges and Solutions in Lift Testing

Like all measurement methods, lift tests have challenges. Let's look at some of them and their likely solutions:

Challenge #1: Overlap between test groups

If you have live campaigns running simultaneously with the test campaigns, control group users could be shown ads they weren't supposed to see.

The key is to have a ‘cool-down' time frame for users in test campaigns. This can ensure they are not exposed to unrelated messages, which might introduce variability.

Challenge #2: Small sample sizes

The right sample size calculations for different significance and power levels are prone to errors.

Automated sample size calculators can make finding the right sample size effortless. All you have to do is input the required significance and power level into the tool, and it does the rest.

Challenge #3: Lack of a unified marketing measurement tool

Marketers are forced to rely on native ad manager tools for lift testing, which are restricted to that platform alone.

The solution is to use an integrated platform like Lifesight, which unifies marketing measurement across channels in real-time, allowing you to measure the true impact of your marketing dollars.

Measure and Validate Marketing Impact with Speed and Precision

As the world moves toward first-party data, marketers are faced with an opportunity of a lifetime: to better understand customer needs and convert them into revenue. Lift testing combined with strategies like MTA and MMM can make you make the right moves and win loyal customers for life.

In the short term, this capability can give you a first-mover advantage amid all the uncertainty prevailing today. Using a proven solution like Lifesight can help you spend more time on strategy rather than data collection and formatting.

Contact us today or book a demo to see for yourself!