The Minimum Data You Should Define by Testing Metric

By Brad

0 comments

Ad Testing

In our last two column we discussed setting your confidence levels and what you need to know about minimum data.

What we haven’t addressed yet is what data points you should assign minimum data levels to based upon your testing metrics.

As an example, if you are testing by CTR, your conversions don’t matter since CTR doesn’t use conversions data in its calculation.

Most metrics have both a required data point  (as that’s the opportunity) and a data point (as the data is secondary to the opportunity) but used to calculate that metric, which would be an optional data point to define.

For instance, click though rate is the ratio of impressions to clicks. You must have an impression to get a click (no impressions will not receive a click). Thus impressions are mandatory but clicks are optional.

In some cases, you might not want to define the optional metric. For instance, let’s say we’re running two ad tests with this data set:

  • Ad 1: Impressions 1000, clicks 100
  • Ad 2: Impressions 1000, clicks 10

In this test, we’re confident that ad 1 is the better ad and has achieve over a 90% confidence interval. However, if we defined a minimum of 25 clicks, we’d still be waiting for results since ad 2 hasn’t hit that number yet. When you define the optional data points, you might wait longer to achieve results if one of your tests is significantly below average (in this case a 10% vs 1% CTR).

With minimum data, every ad in the test should hit the minimum data before you look at the information – not the test combined. As there are many ad rotation options, it is common that not all ads within a test have the same opportunity, and thus each ad should meet the minimum requirements before you examine your confidence levels.

As timeframe is highly important to any test, all metrics should be using a timeframe minimum of at least a week; although, using monthly data works just as well.

Here’s the minimum data that you should define by testing metric:

Metric

Impressions

Clicks

Conversions

Timeframe

CTR

Yes

Optional

Yes

CPA

Yes

Yes

Conversion rate

Optional

Yes

Yes

CPI

Yes

Optional

Yes

ROAS

Yes

Yes

RPI

Yes

Yes

For Adalysis users, if you have defined minimum data amounts (which is optional as the system has prebuilt minimums behind the scenes) we automatically only use the minimum data that applies to each metric, so there is no need to tell the system your minimum data by metric.

Next time you’re running an ad test, consider your minimum data, necessary confidence levels, and then make sure you have the minimum data necessary based upon your testing metrics.

Related Post

Leave a comment

Discover the issues in your Google Ads Account … and how to fix them fast!

Check Your Account For Free!
14 Day Free Trial. No Credit Card Required

PPC Marketing Tips

Subscribe to our blog and receive free, powerful marketing tips delivered directly to your inbox.

Around the Web

How well is your account performing?

Find out, in minutes, what your account score is with our FREE audit report.

Get my account score