A case study of 2,000 Featured Snippets about deduplicating


Updated on July 6, 2020
Topics: ,
18 min well spent

After the chaos around the deduplication of Featured Snippets in the Google search results, I gave in to the itch and analyzed 2,000 Featured Snippets of www.g2.com. I’m happy to be first in line when it comes to screaming about a new feature or decision Google made that hurts websites. But what I noticed this time is that many of us started screaming before having proof of damage. 

Featured Snippet report in AHREFS
Featured Snippet report in AHREFS

So I set out to get my own and found a net positive impact on traffic. Not a substantial one by any means, but noticeable.

In this article, I describe how I analyzed the Featured Snippets and describe the exact results. Mind you, there is a lot of nuance to the analysis and lots of traps you can fall into. I recommend you to read the whole thing thoroughly.

[toc]

In this article, I cover:

  1. Vast differences of Featured Snippets on desktop vs. mobile
  2. CTR differences between Featured Snippet types
  3. Traffic differences between page types

FS Gate summary in 578 words

Here’s what happened: On 1/22, Google got caught at not displaying organic URLs if the page ranks in a Featured Snippet anymore and admitted that it changed something.

SEOs complained (luckily), Google caved and rolled out a whole lotta confusing changes.

Where we are now (2/1): Featured Snippets count as organic results and the ranking URL doesn’t show up anywhere else on the SERP, which doesn’t apply to Featured Snippet thumbnails, refined Featured Snippets, or other SERP Features.

For more information and case studies, check out further reading at the end of the article.

The trap of Feature Snippet impact analyses

Before we jump in, I want to make an important point: comparing the dates before and after 1/24 in search console to determine the impact is not enough. It implies many assumptions that don’t hold true.

Instead, we need to distinguish between:

  • Desktop and mobile because the SERPS look completely different
  • User intent based on query syntax
  • Featured snippet type
  • Country, if you get a lot of international traffic

If you just compare {date before} with {date after}, you mesh all of those in one pott and get distorted results.

In the example for the query “Coursera review” below, you see how the average desktop position is relatively steady while mobile bounces around like an 8-year-old.

Search console report showing ranking differences between mobile and desktop
Search console report showing ranking differences between mobile and desktop for “coursera review”

The desktop result is a regular Featured Snippet without too much sauce.

Google desktop search results for "coursera review"
Google desktop search results for “coursera review”

But on mobile, it’s a Double FS!

Google mobile search results for "coursera review"
Google mobile search results for “coursera review”

That, of course, hurts our CTR and traffic. Mind you that the domain occupying the second FS also serves the image thumbnail on the desktop Featured Snippet. It would be interesting to explore a larger data set and check if that’s always the case.

Also, pay attention to the headings in the mobile FS. On desktop, the FS content is about the quality of Coursera. On mobile, it’s about price and credibility.

Could it be that intent is different on mobile vs. desktop? Is Google’s understanding of the topic is so good that it knows what intent people have when they search on a smartphone versus desktop?

Another impact on click rates and traffic comes from Google’s growing desire to keep people in the search results.

A great example of that is the query “best animation software”, for which we rank in the Featured Snippet on desktop and mobile.

Look at the nice image thumbnails in the Featured Snippet box that attract so much attention. Yeah, they don’t lead you to www.g2.com but to another SERP. A bit unfair, if you ask me because those are results that we at G2 put together. Why wouldn’t they lead to our site where we provide you a ton of information and are specialized in software products?

Desktop

Google desktop search results for "best animation software"
Google desktop search results for “best animation software”

Instead, they lead to a SERP where you get more questions instead of answers. Not a good experience, if you ask me.

Mobile

Google mobile search results for "best animation software"
Google mobile search results for “best animation software”

The same thing happens on mobile, just uglier. When you click on the foldout (see screenshot below), you find a list of entities from the Google knowledge graph (I expect them to have found most of that information on G2 but can’t prove it).

In the “found on the web” box (FOTW Box), Google seems to show the first organic result first but the next result is just a swipe away. A very different click dynamic.

Mobile Foldout

Folded out list featured snippet on Google
Folded out list featured snippet on Google

Another example: “best graph database”

“Best graph database” on desktop

Google desktop search results for "best graph database"
Google desktop search results for “best graph database”

For this one, we rank #1 on mobile, but Google shows a full carousel at the top of the SERPS that leads to…. you can guess it: another SERP.

“Best graph database” on mobile

Google mobile search results for "best graph database"
Google mobile search results for “best graph database”

For me, that means I need to dampen my traffic expectations from queries with the syntax “best {software}”.

How to do a Featured Snippet deep dive

The big question is how to account for all the variables that distort a Featured Snippet analysis. The answer is a Featured Snippet deep dive.

Here’s how it works in a nutshell:

  1. Pull your Featured Snippet queries from a 3rd tool.
  2. Pull clicks, position, and CTR from Search console. If you want to find out what the impact of the 1/24 deduping update on your site was, pull the data before and after 1/24 (ideally 7 days).
  3. Slice and dice, slice and dice.

Now, let’s talk about the details.

First, you need access to a 3rd party tool like SEMrush or AHREFS to pull your Featured Snippet queries. You need to what queries you rank in Featured Snippets for and Search Console won’t tell you (though it should).

I use SEMrush or AHREFS but can also recommend RankRanger.

RankRanger has a cool free tool that shows you desktop and mobile SERP features over the last 28 days.

Desktop

RankRanger report for Desktop SERP Features
RankRanger report for Desktop SERP Features

Mobile

RankRanger report for Mobile SERP Features
RankRanger report for Mobile SERP Features

Exporting Featured Snippets with SEMrush

Featured Snippet report in SEMrush
Featured Snippet report in SEMrush
  1. Go to “Organic Research”
  2. Enter your domain (make sure to choose “subdomain”)
  3. Go to “Positions”
  4. Select “Featured snippet” under “Domain ranks”
  5. Export

Exporting Featured Snippets with AHREFS

Featured Snippet report in AHREFS
Featured Snippet report in AHREFS
  1. Enter your site in Site Explorer
  2. Go to the “Organic keywords” report
  3. Click on “SERP features”, select “Featured Snippet”, and tick “Only linking to target”
  4. Export

When you export Featured Snippets from 3rd party tools, make sure to look at when they discovered the FS. It probably makes sense to filter snippets out that have been discovered later than a month ago.

Second, use the “Search Analytics for Sheets” add-on for Google Sheets or the Search Console API to export queries, page, device (and country, if you get a lot of international traffic).

Search Analytics for Sheets add-on
Search Analytics for Sheets add-on

Make sure to pull one data set before and one after the date the change occurred, in this case, 1/24. Optimally would be 7 or 14 days (even 28 days). Just make sure to factor in that traffic is often different on weekends vs. weekdays, especially for B2C sites.

Third, open all three data pulls (one from your 3rd party tool of choice, two from search console) in Excel and filter out:

  • brand queries
  • queries with less than 5 impressions (just noise)
  • International traffic, if you get a lot
  • Queries with 0% CTR before and after the change date

Design the analysis as you want but make sure to compare CTR, clicks, and position for both, mobile and desktop.

Here’s what my analysis looks like:

My Featured Snippet deep dive spreadsheet
My Featured Snippet deep dive spreadsheet

I created a stupid simple template in Google Sheets for you: FS ANALYSIS TEMPLATE.

Now that you know how to do it yourself, here’s what I found.

The impact of Featured Snippet Deduplication

The Featured Snippet rankings and traffic for www.g2.com dropped on the 22nd when Google deduped snippet URLs and regular results.

On the 24th, our FS traffic started to pick up (mobile: +17%, desktop: +21%) even though our rankings stayed flat.

Two interesting observations:

  1. The first change on January 22nd seems to have impacted us negatively, then the modification on the 24th turned this around.
  2. Ranking positions dipped slightly and stayed flat but traffic picked up.

Desktop Rankings (SEMrush)

Organic ranking report in SEMrush filtered by Featured Snippets
Organic desktop ranking report in SEMrush filtered by Featured Snippets

Mobile Rankings (SEMrush)

Organic mobile ranking report in SEMrush filtered by Featured Snippets
Organic mobile ranking report in SEMrush filtered by Featured Snippets

Desktop Traffic (SEMrush)

Organic desktop traffic report in SEMrush filtered by Featured Snippets
Organic desktop traffic report in SEMrush filtered by Featured Snippets

Mobile Traffic (SEMrush)

Organic mobile traffic report in SEMrush filtered by Featured Snippets
Organic mobile traffic report in SEMrush filtered by Featured Snippets

In the deep dive, I found that our featured snippet traffic improved on desktop but declined on mobile. I see a higher CTR and more traffic after 1/24 on Desktop. On mobile, I see a slightly positive CTR but less traffic. To be fair, I only pulled 584 snippets for mobile – way less than half of the 1,329 desktop snippets.


# of FSCTR after 1/24Weighted CTR after 1/24Traffic after 1/24
Desktop1,329+44%+18%+7%
Mobile5841.4%1%-9%

In the table, you noticed that I added a weighted CTR because you often find some noisy CTR values in Search Console when clicks are low. So, cutting out snippets that got one click after 1/24 will bring your CTR closer to a realistic mean.

The scatter chart below shows CTR changes vs. click changes (before 1/24 versus after).

Most queries don’t show big fluctuations after the FS dedupe. However, there are more points in the upper right part of the quadrant, reflecting the positive traffic impact I noticed.

Scatter chart with Click/CTR changes before and after 1/24 for desktop
Scatter chart with Click/CTR changes before and after 1/24 for desktop

It’s not a huge amount of traffic, though, and there are outliers.

On mobile, there are fewer snippets and outliers, but slightly more points (queries) on the left side of the quadrant.

Scatter chart with Click/CTR changes before and after 1/24 for mobile
Scatter chart with Click/CTR changes before and after 1/24 for mobile

I think it’s also important to differentiate the impact of Featured Snippets between page templates, especially for inventory-driven sites (which I explain in my article about product features for SEO).

Page templates (should) target certain user intents, which is reflected in the types of Featured Snippets Google shows (it’s all connected). If Google shows a list Featured Snippet, users probably want to see a step-by-step process or an overview of something. If you can’t provide that in your content, you’re not meeting user intent. And, since page templates target a user intent and rarely several ones at the same time, you need to look at each separately in your analysis.

As you see for our data, CTR and traffic vary a lot between our product and category pages and on mobile and desktop (see table below).


DesktopMobile
Desktop/products//categories//products//categories/
# of snippets808490395173
Avg. CTR change48%37%1%1%
Adjusted avg.CTR change20%14%1%0%
Click change269-115-125-5

It becomes even clearer when you look at correlations (in the table below). After getting over our issues with correlations 😉, you’ll notice that it’s much more likely for an increase in position to be followed by an increase in desktop traffic. 

That’s not the case on mobile: we see that a higher position can lead to less traffic. The weak correlation is negative.


Desktop CorrelationMobile Correlation
Clicks change/CTR change0.4740.096
Clicks change /Position change0.213-0.138
CTR change/Position change0.369-0.187

Mind you that none of the correlations in this analysis are strong. Not even the 0.47 correlation between click and CTR change would qualify as strong. So, take it with a big grain of salt or two. It’s still interesting to see how this plays out differently on mobile vs. desktop.

As last point, I want to raise awareness for query syntax. In the table below, I compare four query syntax with the modifiers “best”, “alternatives”, “price”, and “review” on desktop with mobile. As you can see for yourself, they’re different in three out of four cases.

CTR changeDesktopMobile
best {software}0%-1%
{software} alternatives1%-6%
{software} price-2%-4%
{software} review-5%-5%

You can certainly take an analysis too far, especially when the traffic we’re talking about here just isn’t that significant. But it still pays off to go deep once in a while to gain a solid understanding of what’s going on.

Search Console data is by no means perfect. In fact, I think Google should indicate where a URL ranks, i.e. in a SERP Featured and in which. But it’s a start and at scale tells a story.

Is FS deduplication a traffic killer?

Let me boldly say: no.

Let me also say that it highly depends on your vertical, brand strength, content quality, device, and type of Featured Snippet you show up for. So, a more refined answer would be: probably not for everyone. 

That does not mean I support the endless SERP mazes Google is creating.

The FS deduplication is a signal to me that Google is more confident in the FS it displays. 

The official talk track is that deduping “declutters the results”.

Some findings in this analysis lead me to believe that Google might have reached a level of understanding that allows it to serve different intent on mobile vs. desktop and do so very confidently.

At a recent Bay Area Search meetup, Gary Illyes mentioned that “Featured Snippets come from an extra algo that bids for that location.

Gary also noted “Featured Snippets relate to the query more than a normal snippet. We try to pick passages that are more complete.

I think that the FS position algorithm doesn’t exist anymore, but the algorithm that matches the query with the content still exists.

What I interpret into that is that we should look for queries within a topic that trigger Featured Snippets and create query-matching content for them in articles or on pages. Not that we haven’t done that before but because organic position #1 is the Featured Snippet now, you can’t rank at the top of the SERPS without query-matching content.

We also must write for mobile and desktop results, based on how the Featured Snippets differ.

Further Reading