How Long Should You Run an A/B Test? [Free Excel Duration Calculator]

How Long Should You Run an A/B Test? [Free Excel Duration Calculator]

As a marketer, you are probably running A/B tests frequently, as there's always something in your campaigns to test and room to heighten and tighten your PPC game.

And if you are dragging with staying consistent with your ad testing, you are not alone. Producing fresh ad copy demands time and thought. It is hard, we get it.

But remember, a smooth sea never made a skilled sailor; hiccups and difficulties you encounter managing accounts will only catapult you to a more fantastic skill-level marketer.

In case you haven't yet got yourself familiar with A/B testing and how it can help you, you can go ahead and check out our post about it. In short, you should definitely utilize this handy tool. (Or a feature if you look at it as Google's built-in feature solely.)

View it as a pool of untapped value for improving your PPC game.

Back to the math now.

The problem with calculating a particular A/B test duration is that you don't necessarily have in-depth knowledge or desire to dive into complex formulas related to analyzing marketing data.

Some formulas are all about the extreme technicality that can be meaningful only to a specialist, which means beyond the rest of us others grasp.

The great thing is that you can skip finding a statistical consultant to give you a hand with calculating how long you should be running A/B tests for.

If you are in A/B testing waters, you already know that there are available, free open sources that offer you access to quickly calculate the duration for your split test.

By the time you read the last sentence of this post, you will know how to calculate it yourself in Excel.

And you can delete bookmarks of those sites that offer free A/B test duration calculation without breaking a sweat.

What do you need for an A/B test duration calculator?

There are 4 components of information that you need to enter to make the calculation:

  • Average daily traffic on your site - if you anticipate a notable change in volume on certain days, note that you should include it when figuring your average.
  • The number of variations to test - have in mind that with more variations you want to test, more traffic you will need. We recommend keeping it simple and have 2 variations, control, and challenging ad.
  • Conversion rate - a current conversion rate of the original ad.
  • Your target improvement - what % difference in conversion rate do you want to detect. If your expected improvement in conversion rate is modest, i.e., if you want to detect even the slightest improvement, it will take much longer. Duration and target improvement are inversely proportional.
Calculating duration of a single A/B test in Excel

Click below to download the calculator:

A/B Test Duration Calculator (Excel Spreadsheet)

If you have downloaded the file and keep it open in a separate tab while continuing to read this post, you might notice that the calculation includes SQRT (Square root) and POWER Excel functions.

But don't fret, you don't need to get into the technicalities of the formula itself.

In this Excel sheet, you have step-by-step written instructions to help you get to the number of days at least needed to run a test.

We say at least since you shouldn't terminate your test beforehand, even if you think you detected a clear winner quickly.

Having this formula divided into several steps, which makes it more granulated and easier to digest, you can always get back to it and project how long you will have to run a test with different figures entered; you can change the conversion rate or experiment with your target improvement. If you expect a lower or higher volume of visitors on certain days that will affect your average numbers, you can modify your average daily visitors to reflect these fluctuations.

For some tests, you will have to wait for an eternity to get results.

Sometimes, there is no guarantee that you will get significant results with an A/B test.

What can happen is that no difference in the performance of control and variation ad will become noticeable, no matter how long you wait, so you will never get a statistically significant result.

For all the people looking to avoid calculations and simply jump over sheets or tables, we have a solution.

If you would rather skip doing the calculation and extracting the needed numbers for your testing, we offer a great tool that saves you time and boosts your overall account performance.

AdOptics helps PPC professionals continually improve Quality Score components, leading to a healthier and better performing account.

By monitoring the ad tests and comparing their performances, our service calculates a winning ad and, at the same time, pushes you to test more and more.

With AdOptics, you don't have to determine how long you should run a single ad split test, as our software does this for you.

And if you would prefer (or need) to be nudged and get reminded to keep testing, hey, we are your go-to solution.

To learn more about how we can help you scale up your A/B split testing game, and improve your overall account, go to our Features section.

Show Comments