A case study of 2,000 Featured Snippets about deduplicating

After the chaos around the deduplication of Featured Snippets in the Google search results, I gave in to the itch and analyzed 2,000 Featured Snippets of www.g2.com. I’m happy to be first in line when it comes to screaming about a new feature or decision Google made that hurts websites. But what I noticed this time is that many of us started screaming before having proof of damage.

So I set out to get my own and found a net positive impact on traffic. Not a substantial one by any means, but noticeable.

In this article, I describe how I analyzed the Featured Snippets and describe the exact results. Mind you, there is a lot of nuance to the analysis and lots of traps you can fall into. I recommend you to read the whole thing thoroughly.

In this article, I cover:

1. Vast differences of Featured Snippets on desktop vs. mobile
2. CTR differences between Featured Snippet types
3. Traffic differences between page types

FS Gate summary in 578 words

Here’s what happened: On 1/22, Google got caught at not displaying organic URLs if the page ranks in a Featured Snippet anymore and admitted that it changed something.

If a web page listing is elevated into the featured snippet position, we no longer repeat the listing in the search results. This declutters the results & helps users locate relevant information more easily. Featured snippets count as one of the ten web page listings we show.January 22, 2020

SEOs complained (luckily), Google caved and rolled out a whole lotta confusing changes.

Where we are now (2/1): Featured Snippets count as organic results and the ranking URL doesn’t show up anywhere else on the SERP, which doesn’t apply to Featured Snippet thumbnails, refined Featured Snippets, or other SERP Features.

For more information and case studies, check out further reading at the end of the article.

The trap of Feature Snippet impact analyses

Before we jump in, I want to make an important point: comparing the dates before and after 1/24 in search console to determine the impact is not enough. It implies many assumptions that don’t hold true.

Instead, we need to distinguish between:

• Desktop and mobile because the SERPS look completely different
• User intent based on query syntax
• Featured snippet type
• Country, if you get a lot of international traffic

If you just compare {date before} with {date after}, you mesh all of those in one pott and get distorted results.

In the example for the query “Coursera review” below, you see how the average desktop position is relatively steady while mobile bounces around like an 8-year-old.

The desktop result is a regular Featured Snippet without too much sauce.

But on mobile, it’s a Double FS!

That, of course, hurts our CTR and traffic. Mind you that the domain occupying the second FS also serves the image thumbnail on the desktop Featured Snippet. It would be interesting to explore a larger data set and check if that’s always the case.

Also, pay attention to the headings in the mobile FS. On desktop, the FS content is about the quality of Coursera. On mobile, it’s about price and credibility.

Could it be that intent is different on mobile vs. desktop? Is Google’s understanding of the topic is so good that it knows what intent people have when they search on a smartphone versus desktop?

Another impact on click rates and traffic comes from Google’s growing desire to keep people in the search results.

A great example of that is the query “best animation software”, for which we rank in the Featured Snippet on desktop and mobile.

Look at the nice image thumbnails in the Featured Snippet box that attract so much attention. Yeah, they don’t lead you to www.g2.com but to another SERP. A bit unfair, if you ask me because those are results that we at G2 put together. Why wouldn’t they lead to our site where we provide you a ton of information and are specialized in software products?

Desktop

Instead, they lead to a SERP where you get more questions instead of answers. Not a good experience, if you ask me.

Mobile

The same thing happens on mobile, just uglier. When you click on the foldout (see screenshot below), you find a list of entities from the Google knowledge graph (I expect them to have found most of that information on G2 but can’t prove it).

In the “found on the web” box (FOTW Box), Google seems to show the first organic result first but the next result is just a swipe away. A very different click dynamic.

Mobile Foldout

Another example: “best graph database”

“Best graph database” on desktop

For this one, we rank #1 on mobile, but Google shows a full carousel at the top of the SERPS that leads to…. you can guess it: another SERP.

“Best graph database” on mobile

For me, that means I need to dampen my traffic expectations from queries with the syntax “best {software}”.

How to do a Featured Snippet deep dive

The big question is how to account for all the variables that distort a Featured Snippet analysis. The answer is a Featured Snippet deep dive.

Here’s how it works in a nutshell:

1. Pull your Featured Snippet queries from a 3rd tool.
2. Pull clicks, position, and CTR from Search console. If you want to find out what the impact of the 1/24 deduping update on your site was, pull the data before and after 1/24 (ideally 7 days).
3. Slice and dice, slice and dice.

Now, let’s talk about the details.

First, you need access to a 3rd party tool like SEMrush or AHREFS to pull your Featured Snippet queries. You need to what queries you rank in Featured Snippets for and Search Console won’t tell you (though it should).

I use SEMrush or AHREFS but can also recommend RankRanger.

RankRanger has a cool free tool that shows you desktop and mobile SERP features over the last 28 days.

Desktop

Mobile

Exporting Featured Snippets with SEMrush

1. Go to “Organic Research”
2. Enter your domain (make sure to choose “subdomain”)
3. Go to “Positions”
4. Select “Featured snippet” under “Domain ranks”
5. Export

Exporting Featured Snippets with AHREFS

1. Enter your site in Site Explorer
2. Go to the “Organic keywords” report
3. Click on “SERP features”, select “Featured Snippet”, and tick “Only linking to target”
4. Export

When you export Featured Snippets from 3rd party tools, make sure to look at when they discovered the FS. It probably makes sense to filter snippets out that have been discovered later than a month ago.

Second, use the “Search Analytics for Sheets” add-on for Google Sheets or the Search Console API to export queries, page, device (and country, if you get a lot of international traffic).

Make sure to pull one data set before and one after the date the change occurred, in this case, 1/24. Optimally would be 7 or 14 days (even 28 days). Just make sure to factor in that traffic is often different on weekends vs. weekdays, especially for B2C sites.

Third, open all three data pulls (one from your 3rd party tool of choice, two from search console) in Excel and filter out:

• brand queries
• queries with less than 5 impressions (just noise)
• International traffic, if you get a lot
• Queries with 0% CTR before and after the change date

Design the analysis as you want but make sure to compare CTR, clicks, and position for both, mobile and desktop.

Here’s what my analysis looks like:

I created a stupid simple template in Google Sheets for you: FS ANALYSIS TEMPLATE.

Now that you know how to do it yourself, here’s what I found.

The impact of Featured Snippet Deduplication

The Featured Snippet rankings and traffic for www.g2.com dropped on the 22nd when Google deduped snippet URLs and regular results.

On the 24th, our FS traffic started to pick up (mobile: +17%, desktop: +21%) even though our rankings stayed flat.

Two interesting observations:

1. The first change on January 22nd seems to have impacted us negatively, then the modification on the 24th turned this around.
2. Ranking positions dipped slightly and stayed flat but traffic picked up.

Desktop Rankings (SEMrush)

Mobile Rankings (SEMrush)

Desktop Traffic (SEMrush)

Mobile Traffic (SEMrush)

In the deep dive, I found that our featured snippet traffic improved on desktop but declined on mobile. I see a higher CTR and more traffic after 1/24 on Desktop. On mobile, I see a slightly positive CTR but less traffic. To be fair, I only pulled 584 snippets for mobile - way less than half of the 1,329 desktop snippets.

In the table, you noticed that I added a weighted CTR because you often find some noisy CTR values in Search Console when clicks are low. So, cutting out snippets that got one click after 1/24 will bring your CTR closer to a realistic mean.

The scatter chart below shows CTR changes vs. click changes (before 1/24 versus after).

Most queries don’t show big fluctuations after the FS dedupe. However, there are more points in the upper right part of the quadrant, reflecting the positive traffic impact I noticed.

It’s not a huge amount of traffic, though, and there are outliers.

On mobile, there are fewer snippets and outliers, but slightly more points (queries) on the left side of the quadrant.

I think it’s also important to differentiate the impact of Featured Snippets between page templates, especially for inventory-driven sites (which I explain in my article about product features for SEO).

Page templates (should) target certain user intents, which is reflected in the types of Featured Snippets Google shows (it’s all connected). If Google shows a list Featured Snippet, users probably want to see a step-by-step process or an overview of something. If you can’t provide that in your content, you’re not meeting user intent. And, since page templates target a user intent and rarely several ones at the same time, you need to look at each separately in your analysis.

As you see for our data, CTR and traffic vary a lot between our product and category pages and on mobile and desktop (see table below).

It becomes even clearer when you look at correlations (in the table below). After getting over our issues with correlations 😉, you’ll notice that it’s much more likely for an increase in position to be followed by an increase in desktop traffic.

That’s not the case on mobile: we see that a higher position can lead to less traffic. The weak correlation is negative.

Mind you that none of the correlations in this analysis are strong. Not even the 0.47 correlation between click and CTR change would qualify as strong. So, take it with a big grain of salt or two. It’s still interesting to see how this plays out differently on mobile vs. desktop.

As last point, I want to raise awareness for query syntax. In the table below, I compare four query syntax with the modifiers “best”, “alternatives”, “price”, and “review” on desktop with mobile. As you can see for yourself, they’re different in three out of four cases.

You can certainly take an analysis too far, especially when the traffic we’re talking about here just isn’t that significant. But it still pays off to go deep once in a while to gain a solid understanding of what’s going on.

Search Console data is by no means perfect. In fact, I think Google should indicate where a URL ranks, i.e. in a SERP Featured and in which. But it’s a start and at scale tells a story.

Is FS deduplication a traffic killer?

Let me boldly say: no.

Let me also say that it highly depends on your vertical, brand strength, content quality, device, and type of Featured Snippet you show up for. So, a more refined answer would be: probably not for everyone.

That does not mean I support the endless SERP mazes Google is creating.

The FS deduplication is a signal to me that Google is more confident in the FS it displays.

The official talk track is that deduping “declutters the results”.

If a web page listing is elevated into the featured snippet position, we no longer repeat the listing in the search results. This declutters the results & helps users locate relevant information more easily. Featured snippets count as one of the ten web page listings we show.January 22, 2020

Some findings in this analysis lead me to believe that Google might have reached a level of understanding that allows it to serve different intent on mobile vs. desktop and do so very confidently.

At a recent Bay Area Search meetup, Gary Illyes mentioned that “Featured Snippets come from an extra algo that bids for that location.”

Gary also noted “Featured Snippets relate to the query more than a normal snippet. We try to pick passages that are more complete.”

I think that the FS position algorithm doesn’t exist anymore, but the algorithm that matches the query with the content still exists.

What I interpret into that is that we should look for queries within a topic that trigger Featured Snippets and create query-matching content for them in articles or on pages. Not that we haven’t done that before but because organic position #1 is the Featured Snippet now, you can’t rank at the top of the SERPS without query-matching content.

We also must write for mobile and desktop results, based on how the Featured Snippets differ.

Further Reading

• CXL Wikipedia case study
• Search Engine Roundtable summary of what happened
• SEMrush experts panel
• Mody Oberstein’s analysis
• Google’s documentation on how Featured Snippets impressions and clicks are counted