Over the past few months at OTTO Digital we have done a series of search ad optimizations for Google advertisers using multivariate content delivery to optimize ad titles, descriptions and visible URLs based on landing page engagement and conversion rate. In addition to improving client ROI we’ve been provided a fascinating look at the effectiveness, or lack thereof, for Google’s Quality Score. Across multiple tests we have found that ads with the highest engagement (as measured by click action on the page) and conversion rates are being displayed less than ads with the highest click-through rates. Substantially less.
Since this is really hurting advertisers ability to generate the highest ROI from AdWords (and my job is to ensure that they do) I’ve attempted to find out all I could about Quality Score.
What do we know about Google’s Quality Score?
Let’s take a look at what Google has revealed about their three different Quality Scores.
Search Ads:
QS1- Minimum Bid: There is a quality score that determines your minimum bid. This is based on these factors of relevance to the keyword:
• Ad Text (as measured by CTR)
• Landing Page (AdsBot reads page text and informs QS based on keyword relevance and ad relevance to page text)
QS2- Position: There is another quality score that determines your ads position. This is based on these factors:
• Ad Text (as measured by CTR)
• Keyword Relevance to Query
Content:
QS3- Content Position: This is the new kid in town. The quality score for content determines you ads position based on these factors:
• Max CPC (as set by the advertiser)
• CTR (on the site you may appear on or on similar sites)
• Landing Page Relevance (to the Ad copy)
Also of interest is how Google’s Ad Optimizer works in relation to Quality Score. In fact, according to Google, prior to the recent Quality Score update, Quality Score was not taken into account at all the past few months if Ad Optimizer was turned off. Now if Optimizer is off Quality Score is indeed a factor. Hmmm.
How is Quality Score affecting advertisers?
It seems many advertisers are up in arms. Especially with this latest update that now includes the landing page factor of contextually-targeted ads. What is clear is that Google is great at measuring the effectiveness of the ad based on click-through but as I suspected at the launch of AdsBot they have no idea or ability to do what they claim, measure relevance of the keyword or ad with the landing page and effectively factor that into ad serving to deliver the best user experience.
Look at these impression levels from a recent multivariate optimization for an AdGroup. There were 9 ads live in the AdGroup and Optimizer was turned off.
Headlines & Impressions
Recipe A-C
Headline: “{keyword}”
Impressions
Recipe A 265909
Recipe B 265045
Recipe C 264784
Recipe D-F
Headline: “Find {keyword}”
Impressions
Recipe D 47093
Recipe E 47289
Recipe F 47066
Recipe G-I
Headline: “The Wait is Over”
Impressions
Recipe G 46518
Recipe H 46795
Recipe I 46579
Remember the aforementioned note about having optimizer off? This same test done before the recent Quality Score update had impression levels almost identical across all tested recipes with the optimizer was turned off.
But here’s the most interesting thing of all. Let’s take a look at the conversion rate for each tested recipe above.
Impressions & Conversion
Recipe A-C
Headline: “{keyword}”
Impressions & Conversion
Recipe A 265909 5.72%
Recipe B 265045 5.09%
Recipe C 264784 5.49%
Recipe D-F
Headline: “Find {keyword}”
Impressions & Conversion
Recipe D 47093 8.59%
Recipe E 47289 5.24%
Recipe F 47066 7.76%
Recipe G-I
Headline: “The Wait is Over”
Impressions & Conversion
Recipe G 46518 9.84%
Recipe H 46795 8.11%
Recipe I 46579 11.76%
Clearly the ads in Recipes G-I are more relevant. Based on the data about 100% more relevant than the ads in A-C that had 6x the impression levels. Even the ads in D-F were significantly better. Why were these six ads served the least? The answer is obvious. Their CTR was much less than the ads in A-C.
The ads in recipes G-I did two things. They drove highly qualified traffic and they converted incredibly well. The advertiser was spending less and getting much better ROI on these ads than the others. Yet their “quality score” is much lower than the other ads. This is because Google does not factor landing page relevance into the quality score for ad position.
Another important point from this study (I will presenting this in its entirety at SES Chicago at the session Ads in a Quality Score World) is that if keyword query relevance of the ads is a factor than we can assume that keyword insertion would benefit quality score (It seems to have been a factor above). But what happens to users when everyone is doing kw insertion? What happens is the description lines become the overriding factor for determining ad relevance, based on conversion rate. How is Google weighing descriptions into quality score? My feeling is minimally, if at all.
At SES Chicago, I will present how description line relevance factors into to ad impressions, CTR and conversion rate. We discovered this only because with multivariate optimization we can take the individual element of the description lines and measure their factor of influence on these metrics.
How should advertisers respond to quality score?
The notion that Google is optimizing for anyone’s performance other than Google’s is misleading and shortsighted. Validating this idea is the lack of transparency for advertisers. For example, we know there are historic factors that affect Quality Score. What are they? Google needs to provide advertisers transparency. Andrew Goodman who’s joining me at SES on the QS panel has just written eloquently about this subject. The second thing that needs to be done is that landing page quality must be a quality score factor for position.
Advertisers deserve better. Google, the experts in user experience should know better. Yahoo and Microsoft, are you listening?
jonathanmendezblog.com 
Leave a Reply