How to Test Responsive Search Ads (RSA) in Google Ads and Microsoft Ads

By Brad

5 comments

Ad Testing, General

There is a common fallacy that you don’t need to test RSAs which are now the default ad type. Your ad is one of the most critical elements of your account since it persuades a user to come to your site and take an action. You need to ensure you’re serving the best possible RSAs

In this article, we will discuss the importance of testing your RSAs and the possible approaches and tools you can use to perform this task.

The Need for RSA Testing

There are 3 primary reasons why testing RSAs is important:

1) RSAs perform worse than ETAs

Without going into all the details again of how ETAs beat RSAs, we know that RSAs rarely perform as well as their predecessors (ETAs).  When migrating from ETAs to RSAs, it’s recommended you do not pause/delete all your ETAs without first ensuring the RSAs can at least match the ETAs performance.  Hence, having a strategy for testing the new RSAs against the existing ETAs is important.

2) Aim of AI algorithm doesn’t match yours

When running an RSA, Google’s machine learning will attempt to find the best performing combination of an RSA so it can serve it most of the time.  Our research indicates Google predominately uses CTR as the metric to decide on which headline combination to serve more frequently than others.  If CTR is not your testing metric of choice, Google’s machine learning is not going to help you achieve your account goals.

For instance, there are 3 RSAs in the below ad group that have all been running for the same length of time. The RSA with the worst conversion rate has more than ten times the impressions of the other ads. Google favored serving that RSA most likely because of its better CTR even though it’s converting at less than half the rate of the others. By pausing this RSA, this ad group increased its conversions by more than 200 in the following 30 days.

Please note: You’re able to see what combinations Google has served so far and how many impressions each of these combinations has. However, Google exposes only the impressions of a combination, and it’s not possible to know exactly which metric Google chooses to favor that combination.  Anecdotal evidence suggests CTR, but this might change depending on Google’s evolving plans.  Therefore, it’s not recommended to rely on the top serving combination as the best one to use without doing further testing.

You can see these combinations in the asset view of an RSA > ‘Combinations’.

3) Machine learning takes a long time

If you create an RSA with 5 unpinned headlines, and Google shows 2 headlines at a time (which is what we see most of the time), you have essentially given Google 20 possible combinations of arranging these into the 2 headlines.  Hence, Google has to serve 20 different ads multiple times to try and find the one combination that performs the best.  Needless to say, this is going to take a while before a winning combination is found, during which time your account performance would suffer.

The below table shows how the scale of this issue is affected by the number of headlines you provide.

For accounts that created new RSAs with 15 unpinned headlines, they are relying on the machine to test up to 2940 combinations.  Assuming each combination needs only 100 impressions to test (which is not enough most of the time), you will need more than 294,000 impressions to discover the better performing combinations.  If you want to be confident in the results, you would need well over 100 impressions per ad combination.  It’s probable that Google has some insights that will speed this up somehow, but the issue of needing a lot of data still remains.

The above is exacerbated further by trying to find a winning combination that includes the descriptions. Providing 3 unpinned descriptions will multiply the above figures by 6, while 4 descriptions will result in 12 times more unique combinations.

To reduce the time it will take the AI algorithms to find winning headline combinations, we can pin one or more headlines which will reduce the total combinations Google iterates through.  We’re essentially shortcutting the machine learning process to speed up reaching conclusions.

The below shows how pinning 1 headline reduces the overall number of combinations the machine has to go through (in the case of showing 2-headline RSAs, which we see most of the time).

As shown above, pinning headlines will greatly reduce the total combinations possible. Hence, employing an RSA testing strategy that involves pinning can help you achieve your goals quicker.

Please note, all pinning discussed in this post is in the context of headlines only.  We do not advise you to fully pin all assets as this will almost always lead to a ‘Poor’ ad strength which will reduce the impressions volume of the fully pinned RSA vs other RSAs in the ad group if the other RSAs are using fewer pins. Please note that there are times you will want to test RSAs where all the assets are pinned.

RSA Testing Strategies

Given what has been discussed, it’s essential that you have a strategy for ensuring the RSAs you create are optimized within reasonable timeframes to achieve the account goals.  Running RSAs without a suitable testing approach can often lead to reduced conversions.

We discuss below 5 different approaches to improving your RSA performance

1) Improve the performance of an individual unpinned RSA ad

This is also known as a fully pinned vs. unpinned ad test.

Scenario 

You have one unpinned RSA ad in an ad group and don’t have any specific headlines in mind that you want to test.  You prefer to rely on the machine learning algorithm to find the winning combination.  Your aim is to find the winning combination with the best performance (in your metric of choice).

Proposed Testing Strategy

Since it will take a long time (given the total combinations discussed above), one approach to speed up the process of finding the winning combination is to look at the current combinations of the RSA ad and use the one with the highest impressions to create a second RSA with the headlines pinned.

By running a second RSA with the headlines of the top combination, you will get to see if this combination will outperform the unpinned RSA and in which metric.  If you’re an Adalysis customer, you’ll have all this data automatically available for you to analyze every day.

Based on the outcome of this test, you can then choose to

  • Keep the pinned RSA and pause the unpinned underperforming version
  • Change the pinned headline to another copy that you believe could perform better
  • Create a third RSA with a different ad copy based on another combination and continue testing

2) Test one set of specific messages vs. the AI algorithm

This is also known as an unpinned vs. partially or fully pinned ad test.

Scenario

You have specific headlines in mind that you want to test to see if they will perform better than what Google’s learning algorithm can achieve.

Proposed Testing Strategy

Run 2 RSAs in an ad group, one that is unpinned while the other has your specific messages pinned.  This is similar to the scenario above but with the second RSA having your specific ad copy instead of the top combination.

With this testing strategy, the pinned RSA may have multiple assets pinned to each position or have assets pinned in only position 1 or 2, depending on what you are trying to test.

3) Test multiple sets of specific messages or themes

These are commonly partially or fully pinned vs. partially or fully pinned ad tests.

Scenario

For marketers who are good ad copywriters,  you are often better than the machine at creating and organizing assets for connecting your message with searchers. In this scenario, you will have specific headlines that you want to test against each other, and you do not want to rely on any machine learning algorithm.

Proposed Testing Strategy

You will want to create two or more RSAs while pinning all of the headlines in each RSA.  You’re essentially creating an ETA vs ETA test.

A variation of this test is to create two RSAs where you are using 2-3 pinned assets for each headline. Essentially, you are jumpstarting the machine learning by putting your ads into a formula:

  • Headline 1: Headlines related to the ad group
  • Headline 2: Call to action or benefit message
  • Headline 3: Supporting headline 2

By creating ads in this manner, you are giving the machine combinations to play with to find your ideal message. However, you are also making sure the ads make sense and contain the type of asset (CTA, benefits, relevancy, etc.) you want in each position.

This variation test is common for accounts with small amounts of data since it can take the machine a long time to learn how to create compelling combinations. By jumpstarting the ad layout, you bypass part of the machine learning phase by giving it some restrictions on the ad serving but still allowing it to find an ideal message for the searcher.

This variation is also common for ad theme testing.  If you have more than one ad theme to test e.g. should your ads:

  • Focus on prices or discounts?
  • Use call to action 1 or call to action 2?
  • Use a benefit or call to action in the second headline?
  • Focus on inventory size or speed of delivery?
  • Use locations in your headlines?
  • Showcase a free trial or why someone should try you versus your competitors?

You would want to create multiple RSAs (maximum of 3 per ad group), with each ad focused on a different theme.  You will pin 1-3 assets to a specific headline in one RSA and then pin 1-3 different assets in the same headline in a second RSA.

Where you will pin these lines will depend on your tests. Usually, headline 1s are good for location tests, and headline 2s are better for price versus discount tests.

Optionally, you can also pin assets to the other headlines. Theme tests can be fully pinned versus fully pinned ads, where each ad uses a different theme.

4) Testing RSAs vs existing ETAs

Scenario

If you have existing ETAs in an ad group, you already have a benchmark as to what the RSA performance should be.  When first creating the RSA in an ad group that has ETAs, the likelihood is the ETAs will outperform the new RSAs added to that ad group.  Hence, your aim should be to improve the performance of the RSA to at least match the performance of the best performing ETA in that ad group.

Proposed Testing Strategy

Create at least one RSA to run alongside the ETA(s) and monitor their performance to determine the statistically significant winner.  You can also use some of the pinning strategies mentioned earlier in this post to speed up the process.

There is a noticeable bias in Google’s algorithms to favor serving RSAs over ETAs.  We frequently see the below scenario where the RSA receives most of the impressions over the ETA even though it’s underperforming in CTR.  Hence, close attention should be paid to ad testing when it involves ETAs and RSAs.

RSA Ad Test 3

When creating the RSAs that will run alongside the ETA, please keep in mind the more assets (headlines and descriptions) you add to the RSA, the more combinations Google has to go through before finding the best performing version.

If you decide to use pinning as a way to speed up the process, one possible approach is to pin 1-3 of the headlines in the top combination (as discussed in strategy 1).

Once you have RSAs that outperform the ETAs you can start testing RSAs vs RSAs in the same ad group as described below.

Tip: check the tools section below for ways to help you quickly create RSAs in ad groups with ETAs.

5) Testing Google’s RSA Ad Serving

It must be stated that you can just create 2-3 RSAs in an ad group, without any pins, and let Google serve the ads. If your ad group only receives a few hundred impressions per month, odds are, you will have very inconsistent metrics, such as CTR and conversion rates, from week to week since Google just doesn’t have the data to test your ads. This is generally not recommended as you often end up with ad serving where your best ad is served the fewest times.

If you do have ad groups with large amounts of data, then Google might get it right. For instance, this ad group receives over 50,000 clicks a month, and after running RSAs for more than 3 months, Google is serving the RSA with the best metrics the most often.

Even when Google does get it right, you will want to pause your losing ads. In this case, when the loser was paused, the ad group saw a nice increase in total conversions. What you should avoid is just creating a few RSAs and then ignoring how each ad performs, as that leads to substandard account performance.

With these types of tests, you are generally taking a themed testing approach, where each RSA is focused around a different theme and then letting Google’s algorithm go to work. However, you need a lot of traffic and patience for this type of testing to work for you. While you can test this way, you will often see substandard results (during the learning period) that could have been avoided by the use of a few pinned headlines.

Tools to help you with the above

Performing the above tasks requires a lot of mundane work that involves creating or editing RSAs in various ways or modifying their assets in bulk (pinning, editing, etc.).  We’ve created a number of tools that make the above tasks much easier and faster to do and hence allow you to establish comprehensive testing strategies that can be managed with a few minutes of your time.  All of the Adalysis RSA tools are accessible by all users (free trial or paid).

Conclusion

The transition from ETAs to RSAs is still ongoing for many advertisers. However, since you can no longer create ETAs, it’s time to understand how to test RSAs. RSAs give you a lot of options for testing ranging from treating them as ETAs (with fully pinned headlines) to providing Google complete control over your asset combinations.

There’s no perfect RSA test. Accounts with large amounts of data can often use more assets and unpinned ads to manage ad tests. Accounts with smaller amounts of data, or the need to have more message control, will utilize partially or fully pinned RSAs.

What is crucial is testing RSAs. Many accounts have created RSAs with 15 headlines and walked away. They are wondering why they are getting fewer conversions. Often, it is due to lower conversion rates of the RSAs combination Google thinks is best.

Your ad is the bridge between a searcher looking for a solution and your website that can convert that searcher into your next customer. Controlling and testing your ads will be an essential part of PPC management, even in the age of responsive search ads and machine learning.

Related Post

5 Comments

  • Ryan

    Great blog post Brad, we’ve all known for a while that RSAs don’t perform as well as ETAs but I’ve not seen anyone write it up like this. Unfortunately I work with a lot of high value/low volume accounts so testing RSAs is really tricky.

    Reply
  • Social Fly

    Thanks for in-depth article. Really good guide.

    Reply
  • John Cammidge

    ‘Many accounts have created RSAs with 15 headlines and walked away. They are wondering why they are getting fewer conversions’ – Very true. I audit many accounts, and 99% of the time every RSA ad is unpinned.

    Reply
  • Ashok Patidar

    Great piece.

    Brad, should we worry about “poor” ad strength? In other words does it matter to have so called “good” or “excellent” ad strength? Does it positively impact the machine learning?

    Reply
  • Leave a comment

    Discover the issues in your Google Ads Account … and how to fix them fast!

    Check Your Account For Free!
    14 Day Free Trial. No Credit Card Required

    PPC Marketing Tips

    Subscribe to our blog and receive free, powerful marketing tips delivered directly to your inbox.

    Around the Web

    How well is your account performing?

    Find out, in minutes, what your account score is with our FREE audit report.

    Get my account score