In SEO, everyone has their own theories about how things do or should work. Some develop their own thoughts and some others prefer to follow existing theories, including recommendations from Google, without validating their ideas.
A hypothesis is nothing more than an idea, an assumption about what one person thinks is the cause of a certain outcome. Hypotheses must be tested through research and experiments to be confirmed as true or not. I see it as the first step to proving impact, and it is essential to understand your successes – or failures. It can start with a hunch or a theory, but ultimately has to be proven.
As SEOs, we can use SEO testing to understand why we may have seen a change in ranking, organic traffic or any other host of SEO metrics.
In this article, I’ll show you how to come up with a hypothesis to test specific issues and how to implement them, becoming more secure in your findings, and giving more answers to your team and clients.
Why do you need a hypothesis?
The short answer is, that without a hypothesis, you’re just kicking a lot of balls and hoping one will reach the goal. Ultimately, without testing a hypothesis, you don’t have a concrete way to genuinely answer SEO questions or explain how or why something may have happened. To know what is working or not, you need to formulate, test and validate your hypothesis.
One question you most likely face as an SEO is, why is traffic down? There might be a lot of reasons: seasonality, PPC taking traffic (own company or competitors), pages being removed or changed, a manual penalty or you actually lost traffic to other organic competitors who may be doing things better than you. The only way to know is to test a hypothesis.
Mark-William Cooks wrote a great article for Search Engine Land, where he talks about a website he created fully on AI. It worked for a few months until Google dropped it completely. He probably had a hypothesis about whether you can rank content written just by AI, without human quality assurance.
The answer he found is: yes you can, but only for a short period. If you want a sustainable business, then the answer is no.
One very important point about creating a hypothesis is that it helps you to stay focused on one idea and it narrows your work. He could have done a manual review on these pages and hired an editor to polish the content. Traffic could increase or stick for longer, but it wouldn’t help him to prove his hypothesis.
Control and test groups
In some cases, you may create control and test groups. The test pages are those that had an intervention, the control didn’t. This can take many shapes or forms, but it’s important to try creating equivalent groups.
That is to say, pages with similar traffic or conversions, search volume, internal links, etc. This is only relevant when you’re about to test something new (e.g. a new internal link module), not when your goal is to run past analyses (e.g traffic decrease), as you’ll see in the examples below.
[Case Study] Optimize organic search traffic using log files analysis
Real-life examples
To move away from theory, I have detailed several examples of hypotheses you can use to test specific things and find more answers instead of raising more questions.
I’m limiting this to broad hypotheses relevant to SEOs everywhere, but if you tend to build things with developers, your hypothesis will get more complex and specific to your company.
I have many examples from my job as an SEO Product Manager, but even if I was allowed to share the details, they wouldn’t necessarily be replicable for a lot of other companies.
Hypothesis one: Traffic decreased because of an algorithm update last month
It’s tempting to “blame” the first thing you see and have an easy explanation, but the truth is usually messier. How can you confirm that a traffic decrease or increase is due to an algorithm update? These are all potential questions or angles that could be analyzed before you reach a conclusion:
- Did the traffic changes happen during a confirmed algorithm update?
- What pages went up or down? Were changes made to these pages?
- Have you seen SERP changes during this period?
- Have you seen competitors’ struggle with volatility during this period?
- Have you kept the same ranking positions?
- Were there any external events, locally (e.g. holidays, celebrations) or globally (e.g. sports events) that could have an impact?
- Are there other changes in the product, such as pricing or checkout flow?
- Can you drill down the changes to one location, device or browser?
- Was there a push on PPC spending that took people away from organic results?
Could your lost traffic be due to a page that has become non-indexable or non-canonical?
A lot of these hypotheses will be collectively analyzed by several stakeholders, that is to say your PPCer will have more budget and competitor insights. Therefore, it could just be a matter of paying attention to what they say or talking to others to rule out a hypothesis until you narrow down your theory and find evidence about the right ones.
Hypothesis two: Conversion increases after redesigning pages
Your new pages are converting higher sales or leads. That’s great, but do you know what is the cause, so you can double-down? These are potential questions to examine:
- Has the traffic to these pages increased as well?
- Were page changes A/B tested? For how long?
- If organic traffic has decreased, is the higher conversion rate paying off?
- Are these pages getting more promotion than before (e.g. Instagram ads, email nurturing) to a specific audience?
- Are users clicking on specific CTAs or are they attracted by a new offering?
Having a good idea about where the growth is coming from allows you to do more of the same. Several things are likely helping with it and knowing exactly what increases your chances to invest more in the right things.
Hypothesis three: Optimizing page titles has a positive impact on traffic
This is a great case in which to create test and control groups. Different from the previous example, here we’re making a change to a group of pages and leaving another group untouched.
The oldest trick in the book: Do you want to know if adding “2024” to a title actually has an impact? Split your pages by traffic and add “2024” to roughly half of them.
- Have the updated pages increased more in traffic vs the control group?
- Have you seen a position change after this update?
- Has the search volume / demand increased?
- Have you seen a change in position or CTR?
- Is there a confirmed Core Update happening at the same time?
I recommend doing this to a large group of pages (as many as you can) so you can reach statistical significance. You can’t claim that something works at scale because you made it happen for five or ten pages.
For more information about statistical significance, Oncrawl has an article about CausalImpact or you can also read this step-by-step guide published on the WTS Knowledge Hub.
Hypothesis four: Adding pages to XML sitemap speeds up indexing
Google has become a lot more selective about what gets indexed over the last few years. Back in the days, we’d take this for granted – we expected pages to be on Google unless they were requested not to be (e.g. noindex), but this has changed drastically.
Let’s say you believe that adding pages to the sitemap is a valid way to get them indexed. This hypothesis can be tested by creating new pages that are only found via your sitemap. This is obviously not an ideal format because real users can’t find the page, but it’s valid from a testing standpoint.
You could also increase the scope by testing a group of pages on an HTML sitemap and another on XML sitemap and see if one or another performs better in terms of indexing – total number of pages or quicker indexing.
Here you’re also required to create test and control groups such as:.
- 50 URLs on XML sitemap
- 50 URLs on HTML sitemap
- 50 URLs are not linked anywhere
In theory, the 50 URLs not linked anywhere shouldn’t be found at all, but you should have a control group just to be sure. Maybe they are linked somewhere you didn’t know and this could be a cause for your test to fail.
Hypothesis five: Adding structured data [type] results in a positive traffic increase
Does structured data (SD) help you to rank higher? This will generate a range of opinions in the industry. Between a “yes”, “no”, “indirectly” and “it depends”, you can only find your answer by testing it.
For structured data in general, here are some specific questions to consider:
- Does adding SD for [price] or [availability] and getting a SERP feature help increase CTR?
- Can you get a Knowledge Panel by adding Person or Occupation SD?
- Does declaring entities (like About or Mentions) help with ranking improvements?
- Should you add SD [type] even though Google says they don’t use them?
- Are SGE results influenced by SD?
Track if the structured data exists on your pages
Structured data is another candidate for CausalImpact (control and test groups) analysis, but impact can also be analyzed on the page itself (e.g. has traffic/CTR increased). Ideally you should update all pages at the same time so it’s easier to measure impact at scale.
Hypothesis six: Adding Internal links have a positive impact on traffic
I’ve lost count of how many times I’ve seen SEOs claiming internal links to be a “secret weapon.” It’s absolutely no secret that internal links are a powerful strategy to increase organic traffic – but how do you achieve this? And how do you know how much impact the strategy had? Some things I’d test include:
- How many internal links do you need to see a traffic increase?
- Does the link location (menu, footer, body) matter?
- Do you need highly contextual links?
As said before, this is an old tactic and certainly has proven its effectiveness. What I attempt to find here is not only if internal links help to increase organic traffic, but by how much, and on a case by case basis.
What belongs (or doesn’t) to a hypothesis
One reason why “it depends” is such a commonplace SEO response, is because all of these things work differently for each website. If you already have great rankings and traffic, there isn’t as much to grow (on the same pages) vs a brand-new website. If you don’t create a lot of new pages, indexing won’t be a problem and so on.
Each hypothesis should be custom-made for a website and scenario. It’s usually helpful to:
- Run one test at a time (for a given group of pages).
- Have a clear hypothesis that can be answered with numbers.
- Sense-check your hypothesis (ideally, outside the SEO bubble).
- Define the period for the test and only reach a conclusion at the end.
Learn, rinse and repeat
I’m a big fan of continuous discovery, something I borrowed from product management principles. Even if a hypothesis has been proven correct and you found, let’s say, a 10% increase in conversions due to your internal linking strategy, or a 50% increase in indexed pages, you shouldn’t stop there.
Can you try an improved version of this initiative? Can you optimize more page titles using other elements? Are these the best SERP features you could have? Keep in mind that even positive outcomes can get better, so keep hypothesizing!
