Get Free Copy Of Our Responsive Search Ads Handbook
Download PDFIn this guide, we will be discussing responsive search ads (RSAs): the best ad layout, the significance of the Ad Strength, the concept of pinning in RSAs and how and if it can affect your ad performance. Finally, we will talk about the benefits of A/B testing your messaging and how to conduct the proper ad tests for RSAs.
In the first chapter, we will give an overview of how ETAs perform in comparison with RSAs. In the second chapter, we will be discussing the concept of Ad Strength and the contributing factors behind it, such as the number of headlines and pinning. In the third chapter, we’ll cover ad testing of RSAs in detail.
If you are still running your ETAs, that’s probably not the worst idea. As our research shows, in many cases ETAs still outperform RSAs in a number of important KPIs.
When we first compared RSAs to ETAs back in 2019, we found RSAs often won for CTR while ETAs often won for conversion rates. That was almost 3 years ago, so in our last ad analysis, we expected to see RSAs win more often across the board as Google’s machine learning has had a lot of time to gather data and create a better algorithm.
Spoiler alert: this didn’t turn out to be this way!
Overall, the win percentages of ETAs are higher than RSAs across all metrics. We feel this is partly due to the RSA format itself which requires more effort from the search marketer to create and manage in more strategic ways than they are used to with other ad types.
We used data from several thousands of accounts to find ad groups that met the following specific conditions:
By using the above criteria in selecting the ad groups, we are confident the conclusions reached in this post are as accurate as could possibly be, and the trends will hold across any large enough group of accounts.
When we look at the number of times one ad type outperformed the other in six different metrics, we find that ETAs won in every single one of the metrics analyzed.
In addition, we compared the impact of rotating ads evenly in campaigns that use manual CPC bidding versus scenarios where optimized rotation was used. We saw no difference in the data of the two sets as far as how often one ad type outperformed the other.
The above results tell us how often one ad type outperformed the other but don’t tell us by how much. To better understand the severity of the difference in performance between the two ad types, we will delve into each comparison in more detail. You will see charts below, for each metric, exposing a better breakdown of the above data.
How to read the below charts
The way the below charts are plotted are as follows:
This was a huge surprise to us. We expected RSAs to at least win by CTR as Google is excellent at optimizing CTR. While ETAs barely won, it was still unexpected.
From the above, we can conclude the following:
This result was not surprising as that has long been the main complaint against RSAs i.e. their inability to consistently convert better. However, the percentage of wins of 67% to 33% was surprising.
From the above, we can conclude the following:
The CPA data shares a lot of trends with the conversion rate data. From the above, we can conclude the following:
ROAS was one of the oddest metrics to work with as there were so many outliers and massive differences in ROAS by ad type. If the outlier was above 100, we lowered it to 100 to make the data easier to read as some of the outliers hit almost 2000, meaning one of the ad types was over 2000 times better than the other ad type.
From the above, we can conclude the following:
We also analyzed the data using a number of different filters e.g. include only ad groups where there were at least 10 conversions per ad, or at least 500 in conversion value, etc. Irrespective of how we sliced the data, the overall trends of huge differences and massive wins in ROAS remained. We are confident that the reason ETAs won so overwhelmingly in ROAS is due to their higher conversion rates.
Conversion per impression (CPI) looks at the ratio between how often an ad is displayed (impression) and how often it receives a conversion. CPI is one of our favorite metrics since it combines both the frequency of clicks (CTR) and how often a click converts (Conversion Rate).
From the above, we can conclude the following:
Conversion value/impression is another favorite metric as it looks at how much revenue an ad brought compared to how often it was displayed. This metric takes into account CTR, conversion rate, and average checkout amount into a single metric. It’s a favorite among e-commerce sites and those that track and bid based upon revenue.
As with ROAS, there were a number of outliers in the 100-250 range, so we capped them all at 100. The outliers were not nearly as extreme as with ROAS, however, a lot of the same types of conclusions can be made as with CPI.
From the above, we can conclude the following:
Overall, ETAs brought in more revenue per impression than RSAs. Much of that is due to the conversion rate differences. We would not want to make this a firm statement, but what we often see is that ETAs are written in specific ways to try and incentivize users to both convert and to have higher checkout amounts through offers like free shipping at certain amounts, discount codes to increase shopping cart basket size, etc.
With ETAs, you know how these ads will be rendered and can make a direct connection from the ad impression to the products. With RSAs, that same control does not always exist. There are still ways of creating RSAs that can make them behave similarly to ETAs which will be covered later in this guide.
Ad Strength is by far one of the most controversial topics around Responsive Search Ads. While some people argue that it is an internal indicator Google uses to tell you they have all the data they need to successfully create all the combinations they want to test, others will reassure you this parameter has a large impact on your CPC, similarly to Quality Score.
Add pinning into the equation and the picture starts looking even more complex, as many people have already noticed that Google doesn’t really like pinned RSA elements since it gives them less ad serving control.
In this section, let’s dive deeper into the data about how Ad Strength is composed and how it affects the performance of your RSAs.
Ad strength is a score Google gives your RSA based upon the total number of assets (headlines and descriptions) and the diversity of those assets within an RSA.
Google has four categories of ad strength:
As you add or edit headlines, your ad strength changes dynamically. In addition, Google will show suggestions to improve your ad strength.
One of the suggestions often seen is not to pin the assets. This is a controversial action, so we’ll take a look at how pinning affects ad strength.
We used data from several thousands of accounts to find RSAs that met the following conditions:
By following the above criteria in selecting the RSAs we are confident the conclusions reached in this post are as accurate as possible.
When we look at ads with Excellent ad strength, most of them have at least 13 headlines, although some had only 8 headlines. No RSAs with under 8 headlines had an Excellent ad strength.
When the above analysis is done for ‘Good’ ad strength, we found one needs a minimum of 4 headlines to achieve a Good ad strength.
Ad strength isn’t only about the number of headlines you have. Google also checks if your keywords are in the headlines and if you have diverse enough content in your headlines. To confirm that ad strength is not only dependent on the number of headlines but rather on the content as well, we examined RSAs that have Poor ad strength and found the majority of them have 15 headlines.
This confirms we cannot enhance the ad strength by simply adding more headlines. We still need headlines relevant to the ad group that are different from each other.
If you have more than 8 headlines and do not yet have Excellent ad strength, look at Google’s recommendations for ad strength. If you see a recommendation to ‘add keywords to the headlines’, and your headlines already have some keywords in them, look at the keywords in your ad group and see if some of them need to be in a different ad group to have more relevant ads associated with those keywords. Our previous blog article looked at RSA creation and how you still need well-organized ad groups even with RSAs and their potential headline diversity.
To examine the relationship between pinning and ad strength, we split our RSA data set into three categories:
We then counted the ads in the above categories segmented by ad strength.
From the above, we can see it’s extremely unlikely to have an Excellent ad strength with any degree of pinning. However, ads that use full pinning can easily achieve a Good ad strength. If pinning is part of your strategy, aim for a Good ad strength (since an Excellent score is much harder to achieve).
If you have multiple RSAs in an ad group, do their ad strengths affect the impressions they get? To conduct an accurate comparison, we considered only ad groups where all RSAs used a very similar pinning pattern (to isolate any impact pinning might have on impressions).
When we examined this data, we see the RSA with the higher ad strength gets more impressions 56.8% of the time. However, the RSAs with the lower ad strength had more impressions 36.6% of the time. We conclude there doesn’t seem to be a clear or strong relationship between ad strength and impressions. Within an ad group, an RSA with higher ad strength will likely, but not always, receive more impressions than the RSA with lower ad strength.
To further analyze the above, we also looked at how the impressions differed with the degree of difference in ad strength and found there wasn’t much of an impact. For instance, the way the impressions differed in an ad group with ad strength of Excellent vs Poor was very similar to when the ad strength was Excellent vs Good.
To examine any relationship between ad strength and CTR or Conversion rate, we again considered only ad groups where all RSAs used a very similar pinning pattern.
In the above chart, we can see that the higher ad strength ads had a lower CTR than the lower ad strength ads in 51.5% of the cases. This number is immaterial and ad strength and CTR do not seem to have any direct correlation.
For the conversion rate comparison above, we once again see that the higher ad strength ads had worse metrics than the lower ad strength ads in the same ad group.
In looking at these charts, there does not seem to be a direct relationship between CTR or Conversion rate and the ad strength. We looked at this data in a few different ways and reached the same conclusion in each scenario.
Given the above where no strong correlation was found between ad strength and the ad’s performance, it seems ad strength is purely a measure of Google’s ability to test the RSA, and that having a high versus a low CTR did not impact the ad strength.
In a lot of our analysis, we often found higher CTRs or conversion rates achieved by ads that use pinning. However, the act of pinning by itself does not have an impact on performance.
If you are pinning well-written assets then, just like writing any good ad, it is common to see higher CTRs and conversion rates due to the better ad seen by the searcher. On the other hand, pinning poorly written assets would typically result in worse-performing ads.
The more unpinned headlines you have, the more variations of ad combinations Google’s machine learning needs to test and optimize. Pinning good headlines often shortcuts the learning process since you are giving the machine fewer variables to work with. Assuming you are only pinning well written headlines, you will almost always see an improvement in your ads performance.
This might seem controversial, but having an Excellent versus Poor ad strength does not seem to affect your account in any significant way. We conclude from our research that:
As Google’s RSA machine learning improves over time, we may see changes to the above conclusions and other account factors. As always, however, the main goal should be to create RSAs that convert regardless of their ad strengths. Because of this, for digital advertisers, testing RSAs is crucial to ensure optimal performance.
There is a common fallacy that you don’t need to test RSAs which are now the default ad type. Your ad is one of the most critical elements of your account since it persuades a user to come to your site and take an action. You need to ensure you’re serving the best possible RSAs
In this section, we will discuss the importance of testing your RSAs and the possible approaches and tools you can use to perform this task.
There are 3 primary reasons why testing RSAs is important:
Without going into all the details again of how ETAs beat RSAs, we know that RSAs rarely perform as well as their predecessors (ETAs). When migrating from ETAs to RSAs, it’s recommended you do not pause/delete all your ETAs without first ensuring the RSAs can at least match the ETAs performance. Hence, having a strategy for testing the new RSAs against the existing ETAs is important.
When running an RSA, Google’s machine learning will attempt to find the best performing combination of an RSA so it can serve it most of the time. Our research indicates Google predominantly uses CTR as the metric to decide on which headline combination to serve more frequently than others. If CTR is not your testing metric of choice, Google’s machine learning is not going to help you achieve your account goals.
For instance, there are 3 RSAs in the below ad group that have all been running for the same length of time. The RSA with the worst conversion rate has more than ten times the impressions of the other ads. Google favored serving that RSA most likely because of its better CTR even though it’s converting at less than half the rate of the others. By pausing this RSA, this ad group increased its conversions by more than 200 in the following 30 days.
Please note: You’re able to see what combinations Google has served so far and how many impressions each of these combinations has. However, Google exposes only the impressions of a combination, and it’s not possible to know exactly which metric Google chooses to favor that combination. Anecdotal evidence suggests CTR, but this might change depending on Google’s evolving plans. Therefore, it’s not recommended to rely on the top serving combination as the best one to use without doing further testing.
You can see these combinations in the asset view of an RSA > ‘Combinations’.
If you create an RSA with 5 unpinned headlines, and Google shows 2 headlines at a time (which is what we see most of the time), you have essentially given Google 20 possible combinations of arranging these into the 2 headlines. Hence, Google has to serve 20 different ads multiple times to try and find the one combination that performs the best. Needless to say, this is going to take a while before a winning combination is found, during which time your account performance would suffer.
The below table shows how the scale of this issue is affected by the number of headlines you provide.
For accounts that created new RSAs with 15 unpinned headlines, they are relying on the machine to test up to 2940 combinations. Assuming each combination needs only 100 impressions to test (which is not enough most of the time), you will need more than 294,000 impressions to discover the better-performing combinations. If you want to be confident in the results, you would need well over 100 impressions per ad combination. It’s probable that Google has some insights that will speed this up somehow, but the issue of needing a lot of data still remains.
The above is exacerbated further by trying to find a winning combination that includes the descriptions. Providing 3 unpinned descriptions will multiply the above figures by 6, while 4 descriptions will result in 12 times more unique combinations.
To reduce the time it will take the AI algorithms to find winning headline combinations, we can pin one or more headlines which will reduce the total combinations Google iterates through. We’re essentially shortcutting the machine learning process to speed up reaching conclusions.
The below shows how pinning 1 headline reduces the overall number of combinations the machine has to go through (in the case of showing 2-headline RSAs, which we see most of the time).
As shown above, pinning headlines will greatly reduce the total combinations possible. Hence, employing an RSA testing strategy that involves pinning can help you achieve your goals quicker.
Please note, all pinning discussed in this post is in the context of headlines only. We do not advise you to fully pin all assets as this will almost always lead to a ‘Poor’ ad strength which will reduce the impressions volume of the fully pinned RSA vs other RSAs in the ad group if the other RSAs are using fewer pins. Please note that there are times you will want to test RSAs where all the assets are pinned.
Given what has been discussed, it’s essential that you have a strategy for ensuring the RSAs you create are optimized within reasonable timeframes to achieve the account goals. Running RSAs without a suitable testing approach can often lead to reduced conversions.
We discuss below 5 different approaches to improving your RSA performance
This is also known as a fully pinned vs. unpinned ad test.
Scenario
You have one unpinned RSA ad in an ad group and don’t have any specific headlines in mind that you want to test. You prefer to rely on the machine learning algorithm to find the winning combination. Your aim is to find the winning combination with the best performance (in your metric of choice).
Proposed Testing Strategy
Since it will take a long time (given the total combinations discussed above), one approach to speed up the process of finding the winning combination is to look at the current combinations of the RSA ad and use the one with the highest impressions to create a second RSA with the headlines pinned.
By running a second RSA with the headlines of the top combination, you will get to see if this combination will outperform the unpinned RSA and in which metric. If you’re an Adalysis customer, you’ll have all this data automatically available for you to analyze every day.
Based on the outcome of this test, you can then choose to
This is also known as an unpinned vs. partially or fully pinned ad test.
Scenario
You have specific headlines in mind that you want to test to see if they will perform better than what Google’s learning algorithm can achieve.
Proposed Testing Strategy
Run 2 RSAs in an ad group, one that is unpinned while the other has your specific messages pinned. This is similar to the scenario above but with the second RSA having your specific ad copy instead of the top combination.
With this testing strategy, the pinned RSA may have multiple assets pinned to each position or have assets pinned in only position 1 or 2, depending on what you are trying to test.
These are commonly partially or fully pinned vs. partially or fully pinned ad tests.
Scenario
For marketers who are good ad copywriters, you are often better than the machine at creating and organizing assets for connecting your message with searchers. In this scenario, you will have specific headlines that you want to test against each other, and you do not want to rely on any machine learning algorithm.
Proposed Testing Strategy
You will want to create two or more RSAs while pinning all of the headlines in each RSA. You’re essentially creating an ETA vs ETA test.
A variation of this test is to create two RSAs where you are using 2-3 pinned assets for each headline. Essentially, you are jumpstarting the machine learning by putting your ads into a formula:
By creating ads in this manner, you are giving the machine combinations to play with to find your ideal message. However, you are also making sure the ads make sense and contain the type of asset (CTA, benefits, relevancy, etc.) you want in each position.
This variation test is common for accounts with small amounts of data since it can take the machine a long time to learn how to create compelling combinations. By jumpstarting the ad layout, you bypass part of the machine learning phase by giving it some restrictions on the ad serving but still allowing it to find an ideal message for the searcher.
This variation is also common for ad theme testing. If you have more than one ad theme to test e.g. should your ads:
You would want to create multiple RSAs (maximum of 3 per ad group), with each ad focused on a different theme. You will pin 1-3 assets to a specific headline in one RSA and then pin 1-3 different assets in the same headline in a second RSA.
Where you will pin these lines will depend on your tests. Usually, headline 1s are good for location tests, and headline 2s are better for price versus discount tests.
Optionally, you can also pin assets to the other headlines. Theme tests can be fully pinned versus fully pinned ads, where each ad uses a different theme.
Scenario
If you have existing ETAs in an ad group, you already have a benchmark as to what the RSA performance should be. When first creating the RSA in an ad group that has ETAs, the likelihood is the ETAs will outperform the new RSAs added to that ad group.
Hence, your aim should be to improve the performance of the RSA to at least match the performance of the best performing ETA in that ad group.
Proposed Testing Strategy
Create at least one RSA to run alongside the ETA(s) and monitor their performance to determine the statistically significant winner. You can also use some of the pinning strategies mentioned earlier in this post to speed up the process.
There is a noticeable bias in Google’s algorithms to favor serving RSAs over ETAs. We frequently see the below scenario where the RSA receives most of the impressions over the ETA even though it’s underperforming in CTR. Hence, close attention should be paid to ad testing when it involves ETAs and RSAs.
When creating the RSAs that will run alongside the ETA, please keep in mind the more assets (headlines and descriptions) you add to the RSA, the more combinations Google has to go through before finding the best performing version.
If you decide to use pinning as a way to speed up the process, one possible approach is to pin 1-3 of the headlines in the top combination (as discussed in strategy 1).
Once you have RSAs that outperform the ETAs you can start testing RSAs vs RSAs in the same ad group as described below.
Tip: check the tools section below for ways to help you quickly create RSAs in ad groups with ETAs.
It must be stated that you can just create 2-3 RSAs in an ad group, without any pins, and let Google serve the ads. If your ad group only receives a few hundred impressions per month, odds are, you will have very inconsistent metrics, such as CTR and conversion rates, from week to week since Google just doesn’t have the data to test your ads. This is generally not recommended as you often end up with ad serving where your best ad is served the fewest times.
If you do have ad groups with large amounts of data, then Google might get it right. For instance, this ad group receives over 50,000 clicks a month, and after running RSAs for more than 3 months, Google is serving the RSA with the best metrics the most often.
Even when Google does get it right, you will want to pause your losing ads. In this case, when the loser was paused, the ad group saw a nice increase in total conversions. What you should avoid is just creating a few RSAs and then ignoring how each ad performs, as that leads to substandard account performance.
With these types of tests, you are generally taking a themed testing approach, where each RSA is focused around a different theme and then letting Google’s algorithm go to work. However, you need a lot of traffic and patience for this type of testing to work for you. While you can test this way, you will often see substandard results (during the learning period) that could have been avoided by the use of a few pinned headlines.
Performing the above tasks requires a lot of mundane work that involves creating or editing RSAs in various ways or modifying their assets in bulk (pinning, editing, etc.). We’ve created a number of tools that make the above tasks much easier and faster to do and hence allow you to establish comprehensive testing strategies that can be managed within a few minutes of your time. All of the Adalysis RSA tools are accessible to all users (free trial or paid).
Here is a quick overview of how Adalysis can help you with managing your RSA:
The transition from ETAs to RSAs is still ongoing for many advertisers. However, since you can no longer create ETAs, it’s time to understand how to test RSAs. RSAs give you a lot of options for testing ranging from treating them as ETAs (with fully pinned headlines) to providing Google complete control over your asset combinations.
There’s no perfect RSA test. Accounts with large amounts of data can often use more assets and unpinned ads to manage ad tests. Accounts with smaller amounts of data, or the need to have more message control, will utilize partially or fully pinned RSAs.
What is crucial is testing RSAs. Many accounts have created RSAs with 15 headlines and walked away. They are wondering why they are getting fewer conversions. Often, it is due to lower conversion rates of the RSAs combination Google thinks is best.
Your ad is the bridge between a searcher looking for a solution and your website that can convert that searcher into your next customer. Controlling and testing your ads will be an essential part of PPC management, even in the age of responsive search ads and machine learning.
Share Around the Web
