There are times when you will run an ad test; but the test is too similar or users don’t react much differently to your test variations and you will never achieve statistical significance.
If you only define minimum data and minimum confidence factors; these types of tests can run for years and you will miss an opportunity to further increase your conversions.
Therefore, you not only want to define minimum data, you also want to define maximum data.
If your ads hit your maximum data and have not achieved your minimum confidence levels, then you need to end the ad test and move on.
There are usually two ways to define maximum data:
- Use 10x your minimum data
- Use a 3 month time frame (assuming your tests are above minimum data)
Defining both minimum and maximum data for your ad tests ensure that you are striving to find actionable information even if that action is to just end a test as the results are not valid and start from a different hypothesis.
For AdAlysis users, we automatically alert you to test results that are above minimum data thresholds and have been running for at least 3 months and have not achieved your minimum confidence levels. There’s no need to worry about defining this information. If you are testing within Excel or another system, ensure that you are defining maximum data so that you don’t have ad tests running that are not going to produce any results so that you’re always striving towards improving your performance.