How the best companies measure content quality

💡
Best practices from UBER, Toast, Angi, Gymshark, WikiHow, Sumo Logic, and Gympass

The most popular SEO strategy is growing traffic through blog content. At some point, especially as a venture-backed company, you want to increase the traffic growth rate from content, and the way to do that is to publish more volume.

However, the inherent risk with higher volume is lower quality. It’s tough to make more at the same level of quality. Think of a local burger chain that becomes a franchise and can’t serve the same tasty burgers anymore. The same principle applies to content. Only few companies achieve high output with high quality.

How?

By measuring the right metrics. Writing is a creative pursuit, but scaling content is a science. We tend to romanticize writing for the web, but good content is the result of a measurable process. Most metrics marketers look at are too output-focused, don’t lead to meaningful action and miss context.

In my experience of scaling content at Atlassian, G2 or Shopify and working with companies like Ramp or Snapchat, I’ve learned that most companies make three critical mistakes with content quality. They:

  1. miss the performance angle
  2. measure the wrong metrics
  3. Don’t tie content quality to business impact

This guide offers solutions to each mistake and a system of quality metrics for each step of the content lifecycle, including the tools to measure them.

In this piece, I’m sharing not only my own experience but also the expertise of

Thank you all for contributing to this piece!

If you don’t have much time, here is the TL,DR:

  1. Don’t confuse performance with editorial content: content for SEO has different requirements than content for positioning or press releases
  2. Focus on controllable, performance-oriented and contextual metrics
  3. For production quality, measure metrics like SEO editor score, Flesch/readability score, or # spelling/grammatical errors
  4. For performance quality, measure metrics like # top 3 ranks, Ration of time on page vs. estimated reading time, inverse bounce rate, scroll depth or pipeline value
  5. For preservation quality, measure performance metrics over time and year-over-year
  6. Don’t forget the bar for high quality is extremely high - most content simply isn’t good enough

Why content quality is critical for Growth

Content quality is critical for three reasons:

1/ We can create content faster and cheaper than ever before with generative AI. The most likely scenario for AI content is a vast amount of mediocre content that nobody cares about. Already today, most content doesn’t get SEO traffic, backlinks or any eyeballs. Generative AI amplifies invisible content - unless the right guardrails are in place.

2/ Content quality has been critical for a long time in SEO since there are only so many search results users click on. But the bar for quality rises every year because the competition isn’t sleeping, and everyone wants a piece of the SEO traffic pie. Most marketers underestimate how good content has to be for a shot at the number 1 spot.

3/ Readers have become a lot more selective about where to invest their attention. The common argument is that attention spans are getting shorter, while the common counterargument is that people binge The Last of Us or watch 3 hour Joe Rogan podcast episodes with Andrew Huberman. The conclusion is that people are much more selective about their consumption because there are so many options. High content quality is the key to mining today’s most valuable mineral: attention.

When I led SEO and Content at G2, we hit a point at which we published 1,000 articles in 6 months, but organic traffic stalled. The content we created was “too shallow”. When we allowed ourselves to invest the necessary time it takes to produce high quality, organic traffic skyrocketed despite publishing frequency going down by over 80%.

The golden lesson for executives is that more content can only equal more traffic (and conversions) if you can keep the quality constant. As soon as you have to sacrifice quality for volume, you’re spinning your wheels or they come off.

In the same vein, low publishing frequency with high content quality also won't allow you to scale SEO revenue with content. The sweet spot is high quality and high volume.

Only high quality and high volume lead to growth at scale

Quality is the key to scaling growth through content. However, most teams don’t measure quality effectively to understand how to keep it high.

3 reasons why most content quality metrics are flawed

Common responses to the question “how do you measure content quality?” are “time on site” or “traffic”. Sometimes, you get a “revenue”. We can do better!

Most common content quality metrics are flawed for 3 reasons:

1/ They’re reactive. A metric is useless when a change doesn’t lead to action (read this again). When traffic or time on site goes down, what do you do? It’s unclear! These numbers don’t tell you much about what’s going on. They’re output metrics. Output metrics are good to know and helpful to report, but only input metrics drive them up (or down) and tell us where things are broken.

2/ They’re not performance-oriented. Content that has the purpose of driving awareness and revenue should be measured with different numbers than content with the purpose of sharing a worldview, position or announcement. A lot of metrics don’t reflect the performance.

3/ They’re not quantitative. Qualitative metrics like comprehension or depth are important because they contribute to a good experience, but they are also very hard to measure at scale.

We need better metrics to create better content. To get to better metrics, we first need to know what we’re optimizing for: performance.

Performance vs. editorial content

When we think about quality, we first have to differentiate between performance and editorial content.

An example of performance content: Shopify’s “guide to starting a business” (link)

Performance content has the goal of ranking in search, driving referral traffic through syndication or attracting backlinks. I call it “performance content” because we can measure impact through hard numbers like MQLs, traffic, ad revenue, etc. Note that some types of content address queries with higher conversion intent than others. Performance doesn’t mean every piece of content needs to generate revenue, but it should influence a performance metric.

An example of editorial content: Marc Andreessen’s essay “it’s time to build” (link)

Editorial content has the goal of generating attention to a narrative. Essays, press releases, product announcements, and what’s sometimes called “thought leadership” falls into this category. We could also call it “content not optimized for search”.

Performance content addresses known problems, editorial content raises awareness of unknown problems. We can define requirements for performance content because known problems typically have search volume, which in return makes it attractive to create content for them. In plain terms, because it’s straightforward to find content Google ranks highly for a topic, we can quantify what good looks like before even writing content.

How to measure content quality

The word quality can be ambiguous, but Seth Godin offers a great definition:

Quality is defined as consistently meeting spec. A measurable promise made and kept. (bolding mine, link)

Measuring content quality is about meeting requirements: topics, sub-topics and pieces of information. Every search query is a question that implies the need for specific information. We can use tools to measure how well a piece of content covers a topic and which information it should provide.

Requirements vary between content types and stages in the content production lifecycle:

  1. Production
  2. Performance
  3. Preservation

The production phase spans the first draft up to a complete piece of content. We can measure the coverage of important sub-topics and how deeply they’re covered through SEO editors (Clearscope, Surfer, Frase, etc.)

In the performance phase, content drives traffic, engagement and conversions (after a brief ramp-up period). The metrics to evaluate at this stage are covered in-depth in the “Metrics for content quality” chapter.

The preservation phase is all about maintaining and improving performance. We can use performance metrics as a signal to refresh or “tune” content to break through plateaus.

Requirements can change over time and vary between topics. Ryan Purtill, Chief Growth Officer at Wikihow, made a good point when he told me “it’s dangerous to view content quality as monolithic”.

Quality metrics need to consider degrees of conversion intent.

  • Exploration (“what is x?”) - low intent
  • Inspiration (“x quotes for y”) - low intent
  • Consideration (“should you buy x”) - medium intent
  • Comparison (“x vs y”) - high intent
  • Conversion (“buy x”) - high intention

The key to an impactful content strategy is considering quality at each phase. As Ryan goes on: “Different types of content can serve different user needs, intents, and mindsets, and thus can serve different business needs. A well thought out content strategy marries those user and business needs.

How Google defines content quality

It would be a mistake to write about performance content and not cover how Google, likely the biggest source of traffic on the web, defines content quality. We can look at two sources for Google’s definition: the Search Quality Rater Guidelines and Google’s SEO Documentation.

Google’s quality framework is EEAT, which stands for Experience, Expertise, Authoritativeness and Trustworthiness. Note that EEAT is an evaluation framework for websites, not an algorithm. However, it’s still useful for us conceptually because we know what good looks like in Google’s eyes.

The rater guidelines mention positive and negative factors for content quality:

Positive

Negative

Originality

Harmful content (misleading/spam/offensive/dangerous)

Accuracy

Copied/auto-generated content that makes no sense

Comprehensiveness

Misleading title

Clear communication

No information about who is responsible for content

Reflection of expert consensus (as appropriate)


Google also measures sitewide criteria, like brand reputation, reviews or ad experience (if applicable) to grade the authority and trustworthiness of a site for specific topics, but it’s very difficult to measure.

Also in Google’s definition, quality requirements vary by site: “The standards for Highest quality MC may be very different depending on the purpose, topic, and type of website.” (definition and bolding added by me)

It’s a little hidden in the long document, but the guidelines emphasize the high bar for quality: “Very high quality MC represents some of the most outstanding content on a topic or type that's available online.

Google’s SEO documentation further clarifies the definition of high-quality content. My summary of the criteria:

  • original information
  • complete description of the topic
  • insights/interesting information beyond the obvious
  • additional value on top of cited sources (don’t just summarize what others say)
  • much better than other results
  • memorable enough to share
  • clear indication of the author (byline, about page)
  • avoiding factual, spelling or stylistic errors
  • no intrusive ads and good mobile experience
  • demonstrating first-hand knowledge and expertise

While these pointers are useful for our conceptual understanding, Google doesn’t provide metrics for content quality. So, how do we turn these requirements into numbers we can measure and improve?

Metrics for content quality

We learned that metrics need to be controllable, performance-oriented and contextual. Measuring them across the content production lifecycle (production, performance, preservation) and considering different types of content (exploration vs conversion) guarantees the highest level of quality.

The following metrics have helped me and other executives to measure and improve content quality at various high-performing companies (get this table as a spreadsheet):


Requirement

Metrics

Goal / benchmark

Tools

Production

(Deep) coverage of important subtopics and key pieces of information

SEO editors score

Better than competing content

  • Clearscope

  • Surfer

  • Frase

  • Semrush Writer Assistant

  • Marketmuse

Production

Easy to read and comprehend

Flesch Score or Readability

80-90 or “good readability”

  • Clearscope

  • Surfer

  • Frase

  • Semrush Writer Assistant

  • Readable

Production

No spelling or grammar errors

# of spelling errors

Zero

  • Grammarly

  • Word/G Docs

  • ProWritingAid

  • Hemingway App

  • Wordtune

Performance

Search and syndication performance

  1. # top 3 ranks

  2. Content ranks in top 3 for target keyword

  3. Referral traffic

  1. min 1

  2. Yes

  3. Growth over time

  • Semrush

  • Ahrefs

  • Sistrix

  • Ryte

  • Similarweb

  • Google Analytics

Performance

Reader engagement

  1. Time on page vs. estimated reading time

  2. Scroll depth

  3. Visitors w/ time on page > 30 sec compared to all visitors

  4. Inverse bounce rate = at least one click during a session*

  5. Sentiment

  1. As close to 100% as possible (min 50%)

  2. Min 30% of users scroll to end of content

  3. Min 30%

  4. Min 5%

  5. Positive

  • Custom dimension

  • Microsoft Clarity

  • Custom dimension

  • Google Analytics

  • Knotch

  • Hotjar

Performance

Converts visitors

  1. Soft conversion rate from organic traffic (newsletter, next page, webinar)

  2. Hard conversion rate from organic traffic (sign-up, buy)

  3. # MQLs

  4. Pipeline value

  5. Sales

  6. Revenue

  1. Min 3%

  2. Min 1%

  3. Growth over time

  4. Growth over time

  5. Growth over time

  6. Growth over time

  • Segment

  • Google Analytics

  • Hubspot

  • Salesforce

  • Shopify

  • Knotch

Preservation

Keeps quality high

Performance metrics over time (rolling 30 days, MoM, Y/Y)

Grow in line with total traffic


*hat tip to Christopher Dean, VP Content at Gympass, for coming up with this metric

How many metrics should you measure? Ideally, all of them! Realistically, not every company can measure every number listed in the table above, and looking at too many numbers can be cognitively overwhelming. If that’s you, pick at least one metric per requirement. Most tools listed above measure several metrics. Semrush, for example, gives you a content score, Flesch score, and tracks rankings.

How should you think about preservation? Measuring preservation is really measuring performance metrics over different time ranges to understand when content performance decays and needs to be updated. Clearscope recently launched its inventory feature, which reruns reports and tells you when performance drops. Another alternative is to rerun SEO editor reports every month to see if content keeps its score.

How can you tie content quality to business impact? Reporting how input influence output metrics is the key to showing how content quality impacts business goals. Higher SEO editor scores, for example, should increase organic traffic and pipeline value, high readability should influence conversion rates, etc. Being able to see and understand this relationship bridges the gap.

Tools for measuring content quality

To quantify content quality, we need tools that show us topic coverage, ranks, readability, user behavior and user satisfaction. Mind you the tool doesn’t matter nearly as much as doing the work. Don’t obsess over it.

SEO editors

Rank trackers

Readability

User behavior

Conversions

User satisfaction

Best-in-class examples of high-quality content

As mentioned in the Google quality rater guidelines, the bar for outstanding content is exceptionally high. The web holds more content than ever before, and it’s growing. Marketers compete against marketers in the search results and try to 1-up each other constantly.

To give you a sense of where the bar is today, the five following content examples show what best-in-class readability and search performance looks like:

Investopedia's guide to cryptocurrency

https://www.investopedia.com/terms/c/cryptocurrency.asp

Investopedia’s content has a very high signal to noise ratio. The articles are comprehensive and get to the point. Note how the sub-headings almost anticipate important questions about the main topic.

Candor's article about salary negotiation strategies

https://candor.co/guides/salary-negotiation

This piece makes it very easy to comprehend with lots of side comments, boxes and notes. Sections have a clear visual separation from each other. Examples come with audio snippets, making the piece more engaging.

OpenPhone's article about business communication

https://www.openphone.com/blog/breakthrough-communication/

This piece on communication feels like a conversation with a friend who knows a lot about the topic. It hits the right balance between casual and informative tone, while providing meaty solutions. The article starts with 7 rules for practical communication and then finishes with 5 unexpected but helpful suggestions for applying them.

Aura's article about child identity theft

https://www.aura.com/learn/child-identity-theft

This article was written by Aura’s CEO and does a good job in structuring the content with lots of bullet points. Videos and boxes with key takeaways make the piece easy to comprehend and skim for key insights.

https://keap.com/product/what-is-crm

Keap’s piece on CRMs has a strong structure with key questions, comparison tables and graphs. It answers key questions, shows examples of a CRM and walks the audience through the step-by-step process of getting started.

Notice how each piece is vibrant and exciting, even though it’s made for performance. We often have this image in our head that search-focused content must be sterile or constraints from SEO editors are limiting. The opposite is often the case.

Other sites mentioned that consistently publish high-quality content are Convertkit, The Well, and Influencermarketinghub.

Conclusion: you can only improve what you measure

Many SaaS companies, online stores and consumer startups want to grow revenue with SEO but can’t leverage user-generated content or a product inventory like marketplaces. They need to create the content themselves, often on a blog or content hub. I call these companies SEO integrators with a marketing-led SEO strategy. The key to success for integrators is publishing the highest possible volume of content with the highest possible quality.

Metrics are the key to high content quality at scale. Marketers can augment them with benchmark articles (hat tip to Amy Lecza) and occasional qualitative research. Collecting a short list of outstanding content and holding it up at the bar to hit helps teams illustrate what great looks like. Focus groups, surveys, prompts or a few gift cards and phone calls for tight budgets can get you to the AHA moment (thanks, Ryan, for the idea).

Trying to quantify everything can seem sterile. As marketers, we love the Mad Men idea of creative, genius writing. But that’s not where performance content lives.

Companies that are serious about growing revenue through content must measure metrics along production, performance, and preservation to be competitive.

The old Drucker quote, “you can only improve what you measure” is worn out but fitting.