Geo-experiments Part 2: How to Deploy and Manage Your Geo Test

This is the second part of our Geo-Experiments 101 series. In the first part, we discussed the basics of geo-experiments, why you should care, and how you can design your first geo-experiment. If you have not read it we encourage you to read part 1.

In the second part, we will be delving into deploying and managing a geo-test, market selection strategies, and shed some light on market size and testing volume.

Let's get started.

Deploying Your Geo-Test: Putting Your Plan into Action

The deployment phase is where your strategies move from the drawing board to real-life action. How you put your plans into action can have a significant impact on the success of your geo test. We'll explore three deployment approaches, each suitable for different levels of testing sophistication.

1. Manual Deployment (Beginner):

At the beginner level of geo experiments, deployment is a hands-on, manual process. You've designed a test where you'll run Google Ads in one set of markets (test group) and pause them in another set (control group). Here's how manual deployment might play out:

• You access your Google Ads account
• You manually select and pause specific campaigns in the control group markets
• You adjust ad spend by manually editing budgets for the test group markets
• You're cautious but still concerned about human error, so you double-check your changes

The manual deployment process in this case is time-consuming and susceptible to errors. Let's say, for example, you forgot to pause a campaign in one control market. This would lead to inconsistent test conditions, potentially skewing your results. Furthermore, because you're running a test in a significant portion of the country, you're taking a substantial risk in terms of lost sales during the testing period.

2. Semi-Manual Deployment (Intermediate)

As you progress to the intermediate level, you aim for more efficient deployment. You export inclusion and exclusion lists to ad platforms for your campaigns, which means your process looks like this:

• You prepare inclusion and exclusion lists for both test and control markets
• You export these lists to an ad platform like Google Ads, applying the changes across campaigns
• The changes are made more quickly and with less chance of manual errors

With semi-manual deployment, you've streamlined the process and reduced the risk of manual mistakes. However, you're still overseeing the deployment process, and human intervention is involved.

3. Automated Deployment (Advanced):

The pinnacle of sophistication in deploying geo tests is the fully automated approach. In advanced geo-experiments, you leverage automated deployment technology to ensure precision and efficiency. The key driver behind this automation is the use of Application Programming Interfaces (APIs).

• You set up your tests in a geo-testing platform like Lifesight equipped with automation
• You define your test parameters and schedule the tests with a few clicks
• The platform interfaces with an ad platform like Google Ads through APIs, making changes to your campaigns on scheduled dates.
• The entire process is executed swiftly and accurately

In the advanced deployment approach, you launch multiple geo tests in different regions almost simultaneously. With the aid of automation, you avoid the risk of human errors entirely. This means you can focus on running and analyzing your tests with confidence, knowing that the deployment process is seamless and error-free.

The business impact becomes significantly minimized with automated deployment. You can efficiently test various markets without the need for extended cooling-off periods or concerns about substantial sales losses thereby helping you to not only save time but also enhance the reliability of your geo-experiments.

To summarize, here's a table listing the three deployment methods along with their pros and cons:

A Note on Market Selection Strategies

Selecting the right geographic areas for your geo-experiments is like choosing the perfect ingredients for a recipe. Just as the right mix of ingredients can make or break a dish, the choice of test markets can significantly impact the success of your advertising experiments. In this section, we'll explore the importance of market selection and the strategies to ensure your test markets accurately reflect national performance.

The importance of choosing the right geographic areas

Why does market selection matter so much you ask? Well, the answer lies in the heart of what geo-experiments aim to achieve: measuring the true, incremental impact of your advertising efforts. To obtain meaningful insights, your test markets must be representative of your broader target audience.

If you choose the wrong geographic areas, your results may be skewed, leading to inaccurate conclusions.

Firstly, the primary goal of geo-experiments is to make data-driven decisions about advertising investments. If your test markets don't mirror the behaviors of your national audience, the insights derived may not be valid. Inaccurate results can lead to misguided budget allocation.

Furthermore, marketers rely on the results of geo-tests to make strategic decisions. Choosing test markets that are unrepresentative can lead to false conclusions. These misinterpretations can result in wasted ad spend, missed opportunities, or incorrect strategy shifts.

Finally, conducting geo-experiments consumes resources, including time and budget. By ensuring the right market selection, you maximize the utility of these resources and reduce the chances of resource wastage due to inaccurate or unrepresentative tests.

Market Size and Testing Volume

For running geo-experiments, the size of your test markets is more than just a logical detail. It's a critical factor that can significantly impact the accuracy, efficiency, and overall success of your advertising experiments.

Why does test market size matter?

The size of your test market matters for several reasons, however, the most prominent ones are

1. Representativeness
2. Resource management
3. Testing volume

For your geo-experiments to provide meaningful insights, your test markets must closely resemble your national audience. Larger test markets increase the risk of introducing non-representative elements that can skew your results.

Also, running experiments in larger markets can be resource-intensive. It requires allocating a significant budget and workforce to ensure smooth execution making efficient resource management crucial for long-term sustainability.

However, testing volume is the most important reason why market size matters. The size of your test markets directly affects how many experiments you can run concurrently. Smaller test markets provide the flexibility to test multiple hypotheses simultaneously, helping you gather insights more quickly.

Ensuring you run multiple tests concurrently

Running multiple tests concurrently is a strategic advantage that can significantly enhance the efficiency of your geo-experiments. Here's how you can do it:

1. Avoid contamination

Contamination occurs when two or more tests overlap in the same markets, making it difficult to attribute results to a specific test. To run multiple tests simultaneously, avoid overlapping test markets.

2. Advanced tools

Leverage advanced tools like Lifesight that support concurrent testing. Such a tool often includes automated design, deployment, and reporting features that streamline the testing process.

3. Segmenting your audience

Consider segmenting your audience into distinct groups. By doing so, you can run different tests for each segment concurrently, allowing for a more comprehensive testing approach.

In the next part ...

Well, that was the end of the second part of our Geo experiments series. We've taken you through the journey from manual deployment to semi-manual deployment and finally, the advanced world of automated deployment.

Now, we're excited to guide you into the third and final part of our series, where we'll explore the analysis and reporting phase. Here, you'll discover the secrets of measuring incrementality, validation, maximizing business impact, and why continuous testing is crucial.

Meantime if you want to see how you can design a geo experiment on Lifesight, sign up for a free demo today.