If you’re unsure whether making a change to your website will positively or negatively affect SEO, running an SEO test might be the solution.
But when should you do SEO testing, and how?
In this guide, you’ll learn:
SEO testing is the process of making changes to your website and measuring the impact on organic search traffic or rankings.
Here are four benefits of running SEO tests:
Known ranking factors aside, there’s a lot of “it depends” scenarios in SEO. Just because someone saw a positive impact after making a change doesn’t mean things will play out the same way for you. It’s better to test it on your website.
Rolling out SEO changes can absorb a lot of time and resources. If that change doesn’t have the desired effect, it’s a waste of energy. Testing tells you whether something is likely to have a positive impact in less time using fewer resources.
Everyone has an opinion on what does and doesn’t work in SEO. Often the quickest way to settle disputes is to test them. Fail to do this, and the person with the loudest voice tends to dictate direction—which isn’t always for the best.
If there’s one thing worse than spending time and effort rolling out an SEO change that has no impact on SEO, it’s doing the same for something that negatively impacts SEO. By running an SEO test, you can limit any negative outcome to a small subset of pages.
There are three main types of SEO tests.
Serial testing is where you change all of the pages on your website at once and observe the outcome. We wouldn’t recommend this for three reasons:
Time-based testing is where you make a change to one page and see how it performs. We don’t recommend this type of SEO testing because it’s unwise to assume causation with a sample size of one.
A/B testing (or split-testing) is where you change some pages but not others. You then compare how the changed pages fare against the unchanged pages. The group of unchanged pages is known as your control group, and the group of changed pages is known as the variant group. We recommend this type of SEO testing because:
SEO testing probably isn’t right for you unless your website gets a significant amount of organic traffic. We’re talking tens or hundreds of thousands of organic visits per month. There are two reasons for this:
If your website gets plenty of traffic, improving rankings or increasing traffic by a small percentage can significantly impact your business—and this is where SEO testing makes sense.
Follow these seven steps to get started with SEO testing:
A hypothesis is a prediction. It’s where you decide what you’re going to change and how you think it’ll affect your website’s SEO visibility.
Here’s a simple formula for forming a hypothesis:
[change] will lead to [effect] on [types of pages]
Here’s an example:
[adding short descriptions] will lead to [a 10% improvement in organic traffic] on [patio furniture ecommerce category pages]
You can see that we’ve outlined what we’ll change, the intended outcome, and the types of pages we’ll use for the test.
Make sure to choose a group of pages that have a lot in common when doing this—like blog posts or e‑commerce category pages—to minimize variables that can screw up your test results. Your hypothesis should also be an educated guess, not just something random.
It’s best practice to only run tests with pages that get a decent amount of organic traffic. This is because you’re not going to learn much from a page that gets little or no organic traffic. It’ll just skew your test results.
You can find pages with decent organic traffic in Google Analytics:
If you don’t use Google Analytics, you can do the same thing using Ahrefs’ Site Explorer.
Download the results and clean them up where required. These are the pages that you’ll use for your test.
You’ll be running an AB split-test, so you’ll only need to make changes to a random subset of pages. The remaining pages will form your control group.
If you’re using a dedicated SEO testing tool, it may handle random sampling for you. However, if you’re doing things manually, the easiest way to pick a random sample is using the “Randomize range” function in Google Sheets.
Here’s how to do it:
This randomizes the order of URLs, meaning you can now use a percentage of them from the top as your variant group.
Knowing how long to run a test for can be difficult, but it’s basically as long as it takes to gather a decent amount of data. This could be anywhere from a few days to a few months. It depends on how much traffic you get and how many pages you have.
Here are a couple of pointers:
If you’re not sure how long to go for, I recommend starting with a month. You can always leave the test running for longer if you need more data.
Before you make changes to your site, you need a way to track results. The easiest way to do this is to use an SEO testing tool. It connects to your Google Search Console account and tracks results for you.
But you can track results manually if you prefer.
If you want to test changes in organic traffic, use Google Search Console or Google Analytics. Just compare average differences in traffic to both groups for the period before and after the changes were made.
If you want to test changes in CTR, use Google Search Console and compare average changes.
If you want to track changes in organic rankings, use Google Search Console or a rank tracking tool like Ahrefs’ Rank Tracker. The latter is arguably the better option because it gives you actual ranking positions—not just averages.
To set this up in Ahrefs’ Rank Tracker, add and tag two batches of keywords:
You can then see average ranking positions for the keywords in each group in the Tags report:
It’s finally time to implement your AB test. Here are a few pointers for doing this:
If you’re using a testing tool, it’ll probably produce some nice graphs for you to analyze once the test is over:
If not, you can evaluate the outcome of your test manually using Google Analytics (if you’re testing for changes in traffic), Google Search Console (if you’re testing for changes in traffic, rankings, or CTR), or a rank tracking tool (if you’re testing for changes in rankings).
If you’re testing for changes in rankings and everything’s set up in Ahrefs’ Rank Tracker, as shown in step five, you can see changes in tracked keyword visibility since the start of your test in the Tags report. Just choose the timeframe.
Everyone should run different SEO tests depending on their SEO goals. But let’s look at a few examples of common tests to drill the process home.
As title tags show up in the search results, they can affect your pages’ clickthrough rates (CTR). If you use the same formula for all product or category pages like most ecommerce sites, testing the impact of a different formula on CTR is pretty straightforward.
Take a bunch of e‑commerce category pages that get a decent amount of organic traffic, take a random sample, then change the title tag formula on those pages.
For example, let’s say that your current formula for product pages is:
[Product name] | [Brand]
You might want to test whether adding “Free delivery” improves CTR:
[Product name] — Free Delivery | [Brand]
Use Google Search Console to compare the average CTR for pages in both groups for the period before and after the changes.
You’ll probably need to export this data to a spreadsheet or Google Data Studio as the filters in Search Console leave a lot to be desired.
Many SEO professionals swear by content optimization tools—but many others don’t. If you’re in the camp that thinks there might be something to this approach, why not test it?
Take a bunch of blog posts that get a decent amount of organic traffic and rank in the top 10 for their main target keywords, take a random sample, then “optimize” those posts with a content optimization tool.
Here’s how to find such blog posts in Ahrefs’ Rank Tracker:
Use Ahrefs’ Rank Tracker to compare the average visibility for tracked keywords in each group on the test start and end dates as explained in step five.
Remember to tag keywords as control and variant groups in Rank Tracker at the beginning of your test.
Ranking in featured snippets usually drives more clicks. So if you have blog posts targeting and ranking for keywords that trigger featured snippets, why not test your ability to win them?
Take a bunch of blog posts that get a decent amount of organic traffic and rank in the top 10 for keywords that trigger featured snippets, take a random sample, then add snippet-friendly content to those posts.
Here’s how to find such posts in Ahrefs’ Site Explorer:
Use Ahrefs’ Rank Tracker to compare the number of owned featured snippets for tracked keywords in each group on the test start and end dates.
Remember to tag your keywords as control and variant groups in Rank Tracker at the start of your test.
To keep this article as simple as possible, I intentionally avoided the issue of statistical significance. This can get real complicated real quick, but it’s basically a way to prove that the results of an SEO test are reliable and that there’s a high probability of a causal relationship between the change and outcome. In other words—that whatever happened (or didn’t happen) is unlikely to have been by chance.
I’m not currently aware of any SEO testing tool that calculates this for you. So let me reiterate my earlier advice: don’t run SEO tests unless your website gets plenty of traffic.
If you’re running tests and making conclusions after only a few hundred visits, chances are the results aren’t going to be statistically significant.
Got questions? Ping me on Twitter.
Source: ahrefs.com, originally published on 2021-08-17 17:42:27