New Google Ad Format Only Changed User Behavior For 3 Months

Back in November 2013, reports surfaced of Google testing a new user interface for search results shown to desktop users. On March 13th of this year, Google’s new search results design officially went live for everyone, marking a big change in how ads are displayed in search results.

Folks at UK-based digital strategy company Fused7 were among the first to report this widespread change, and they captured these great before and after shots that show the change clearly. Mainly, the change made paid ads blend in a lot more with the natural search results. Not surprisingly, Google made a change that made users click more on the ads.

But the gain to Google in ad clicks has by now come undone for some of the most common ads we run at Groupon. As the chart below shows, users have reverted to the norm and now scroll past the ads to click on the organic search results at pretty much the same rate they did before the change.

7-day moving average of SEO and SEM clicks for select brand keywords

At the far left of the chart, you can see the ratio of natural search clicks to paid ad clicks for a set of branded keywords we track (in this case, keywords that include the word “groupon”). These keywords send users to our home page, city pages, and other places where we have heavy brand search.

Before Google’s change, we got about 15% more clicks on natural results, aka SEO (shown in orange above), than we did on paid results, aka SEM (shown in blue above).

Note: The chart shows a 7-day moving average which makes it easier to read without the weekly cycles our business encounters, but it has the effect of showing the impact a few days later and a little slower to reach full impact than reality. In reality the time it took for Google to ramp this change from very few users to near 100% of users, and the subsequent drop in SEO and rise in SEM, happened very quickly with most of the impact in about one day.

By March 17, the 7-day moving average of clicks for SEM was 75% higher than SEO for our test basket of keywords.

At this point, outlined by the purple section of the chart, we started doing some testing of SEO vs. SEM. This is something we do from time to time, inspired by, and using similar techniques as, eBay had published back in 2013. We did this to confirm that the drop in SEO really was due to Google’s UI change and to measure SEM’s incrementality in this new world Google had created.

Turning SEM off for periods of time on different days caused SEO to increase. When we stopped this testing and once again left SEM on at a constant rate, SEO again fell to where it was on March 17th.

But then starting about April 20th, we began to see this change wrought by Google magically undo itself. By June 10 (3 months after the change) users had pretty much reverted to their normal behavior and were once again clicking on natural search results (SEO) and paid results (SEM) at the same rate that they were back in early March before Google’s change.

This Raises A Few Interesting Questions

1. Did Google know this was going to happen or was it disappointed when ad revenue fell back to where it started?
2. Does Google know more than we do (ha, silly question), and is it still making more money in other areas of search with this change?
3. Why do users really care so much that they are willing to scroll past the ads and seek out the natural search results?

I can’t really answer (1) and (2); but I have an opinion on (3), as I’m sure many readers here will.

Users scroll past ads because natural results are better. To rank #1 for a natural search result, you really have to be relevant, and Google does a great job of ensuring that.

To have an ad placement, however, you don’t need to be as relevant, and users are routinely disappointed by what they find behind ad clicks.

Who hasn’t had the experience of searching for some very specific thing, clicking on an ad that echoes back your search keywords in a way that makes you believe the page it will lead to is exactly relevant to your search, and then arriving on the page and being horribly disappointed?