Root-cause analysis traps: where is the Smoking Gun?

Instead of searching for the Smoking Gun, embrace a problem-solving approach based on risk-mitigation, continuous improvement, adaptation, collaboration, and learning.

I work with two types of clients: those who want to grow and those who want to turn a negative trend around: a drop in organic traffic, conversion rates, or non-paid revenue. They bring me on to find "The Smoking Gun”, a single issue that needs to be fixed to solve the problem.

But there rarely is a Smoking Gun. With the exception of a few technical errors, Organic Growth problems are multi-faceted.

There used to be a time when algorithms were simple and users easy to track. Smoking Guns were easy to spot, especially in SEO: algorithm updates, link penalties, technical issues.

But today’s world is not that simple anymore. Problems with multiple causes are harder to understand, solve and measure. The reward is long-term gains. We’re not making the car faster by finding the nail in the tire.

Smoking Guns in SEO, CRO and Email Marketing

There are a few technical issues that can hold a business back.

Smoking Guns in SEO:

  • Pages are accidentally no-indexed or excluded from crawling in robots.txt
  • Pages have the wrong canonical tag
  • Faulty hreflang tag implementation
  • Rendering issues
  • Large amounts of orphaned pages, 4xx and 5xx

Smoking Guns in conversion rate optimization:

  • Broken CTAs
  • Broken conversion funnel
  • Long loading times

Smoking Guns in Email Marketing:

  • Broken email layout
  • Broken links in emails
  • Campaigns that get caught by spam filters

But skilled professionals can often find them with a few days of auditing and develop systems that catch them automatically. They let them occur only once. So, long-term gains don’t come from fixing single-occurrence problems.

Then why is it that many marketers and executives are obsessed with Smoking Guns?

Obsessing over root-cause analysis

The approach to finding Smoking Guns is root-cause analysis. A causes B. A common approach is the 5-whys. The Silicon Valley school of Growth, which I would consider myself a graduate of, is obsessed with root-cause analysis. But not every problem has a single cause.

I get it. I love system thinking and taking an engineering perspective of dissecting every problem into its smallest parts. But algorithms have become too sophisticated, and we’re no longer getting the same amount of data. We can build revenue engines on top of data from platforms like Google, Meta, Amazon and web tracking, but we can’t understand how much exactly every single part of the machine contributes to the outcome anymore.

Take the concept of thresholding, for example, which is commonly used in social and search algorithms. Certain outcomes, like higher visibility, only come into effect when several criteria are met. For example, content needs to meet certain criteria for Google to consider it helpful and rank it in organic search [link]. The opposite is a binary factor like Twitter filtering out NSFW content [link]. The challenge of dealing with thresholds is that each criterion can have different weights, which is very hard to comprehend and reverse-engineer for human brains.

You would think over time, we get more data because our tools improve, but the opposite is the case. Google heavily filters traffic in Search Console, stopped sharing the keyword referrer in 2011 [link], pushes attribution models to intransparent data-driven attribution, and forces advertisers to rely on machine learning for Google ads keyword-targeting.

The conditions in Organic Growth are harder for root-cause analysis, but the point is not that it's useless. The point is to check for the few Smoking Guns that still exist and then switch to a different mental model.

5 alternatives for Root-Cause Analysis

Root-cause analysis can be an effective approach, but when problems are too complex, it can be misleading.

I want to offer 5 alternatives:

  1. Risk-based approach: Focus on identifying, assessing, prioritizing and mitigating risks that can lead to problems in the future.
  2. Continuous improvement: Regularly evaluate and update processes and systems based on feedback, lessons learned, and best practices.
  3. Adaptive management: Embrace an iterative approach to problem-solving, where solutions are tested, evaluated, and refined over time based on changing circumstances or new information.
  4. Collaborative problem-solving: Solve problems collectively across multiple disciplines (e.g., engineering, design, data) with diverse perspectives and expertise.
  5. Learning from successes: Analyze successful projects to understand contributing factors and apply insights to other areas of the organization.

Most organizations can run a mix of all 5 but should focus deeply on one approach.

The way to detach yourself from root-cause obsession is a mental shift from expecting a certain result from all Organic Growth activities, which is possible in areas like Product or paid acquisition, to making space for learning and testing when setting goals.

Companies that embrace one of the 5 alternatives to root-cause analysis

  • have ongoing experimentation programs
  • set learning goals next to outcomes
  • build, share and groom a repository of insights (successes and failures)
  • plan capacity for discovery
  • run on high shipping frequency
  • cut Organic Growth teams slack but also hold them accountable for rigorous execution

The most successful companies at Organic Growth acknowledge that activities like SEO don't lend themselves well to root-cause analysis. They stop searching for the Smoking Gun.