Welcome to the #SEOchat, where industry experts and enthusiasts come together to discuss the ins and outs of Search Engine Optimization (SEO). Today, we delve into a topic that often remains shrouded in mystery—SEO testing. For many marketers, SEO testing may seem like an intimidating realm filled with complexities and uncertainties. However, fear not! In this blog post, we will demystify SEO testing and uncover its importance in the world of SEO.

Understanding SEO Testing

To kick things off, let’s address some common misconceptions surrounding SEO testing. One of the reasons behind the hesitancy to embrace SEO testing lies in its conflation with Conversion Rate Optimization (CRO)-style A/B testing. It’s essential to clarify that these are distinct approaches with different goals.

According to the insightful contributions of the #SEOchat participants, the complexity of SEO testing and the challenges in discernibility, validity, and resource allocation are some factors that hinder widespread adoption. However, they also emphasize the fundamental role of testing in understanding the impact of SEO activities and improving overall performance.

Simplifying SEO Testing

Fear not, for our experts are here to share some invaluable advice for anyone looking to dip their toes into the world of SEO testing. Logan Bryant and Darth Autocrat suggest that getting started with SEO testing is easier than it seems. Logan advises us to “Just play with it.” A few simple tests, including reversions and repeats, can provide valuable insights. On the other hand, Darth Autocrat reminds us that testing is at the core of good SEO, and it’s all about the do-measure-learn process.

Debunking Myths

The #SEOchat participants also debunk various myths surrounding SEO testing. From the misconception that it’s overly complicated to the belief that observational studies are a better alternative, they set the record straight. Additionally, they shed light on the different types of SEO tests, such as time-based tests, A/B split testing, and multi-URL testing.

Embracing Bayesian Measurements

In the realm of data science, the debate between “Frequentist” and “Bayesian” measurements is of great interest. The experts elaborate on how Bayesian measurement can provide valuable insights, especially in cases where traditional statistical significance may be lacking.

Unveiling the Best SEO Testing Practices

Throughout the #SEOchat discussion, our participants share valuable tips and best practices for successful SEO testing. They emphasize the importance of tangible results and the need to demonstrate the potential impact before gaining buy-in for broader changes. Furthermore, they recommend resources and courses that can guide both newcomers and seasoned SEO professionals.

Join the Journey of SEO Testing

In this blog series, we invite you to join us on a journey of discovery, experimentation, and growth in the realm of SEO testing. Whether you’re a seasoned SEO expert or a curious beginner, this series aims to equip you with the knowledge and confidence to navigate the world of SEO testing effectively.

Stay tuned for our next post below, where we’ll dive deeper into the types of SEO tests, practical case studies, and the tools you need to kickstart your own SEO testing initiatives. Remember, SEO testing is not a daunting beast; it’s a powerful ally in your quest for SEO success!

A1: I think there are a few reasons for this. 1) Marketers conflate SEO testing with CRO-style A/B testing, which is a whole different beast. 2) SEO testing sounds like an intimidatingly large skill to learn (not true) 3) SEO testing hasn’t been part of the zeitgeist

 Logan Bryant

I feel like it’s because we feel SEO testing is verrrry complicated and either requires too much effort or time

Mordy Oberstein 

Discernibility Identifying/attributing a change can be a challenge Complexity As you can’t always “undo and see”, or run “true” comparisons, it’s not simple Validity Testing for concreteness takes multiples, and time Authorisation Resources and risks deter

Darth Autocrat (Lyndon NA)

A2: To that person, I would say SEO testing is foundational to everything moving forward in SEO. It doesn’t have to be complex. At it’s core, testing is just the do-measure-learn process. To start, just change one title tag and wait a few weeks to see what happens in GSC

Logan Bryant

I’d say “just play with it”. Just a few simple tests, including reversions and repeats, can give you insights into what may be worth investing in, or not.

Darth Autocrat (Lyndon NA)

A3: As long as there is SEO, the testing portion will remain valuable. When SEO dies, then SEO testing will follow. This is because testing is fundamentally how we learn whether or not the SEO activities we are doing have material impacts. All of us are SEO testing.

Logan Bryant

Testing and seeing how Google responds is synonymous with doing good SEO

Mordy Oberstein

A3: I think it will become a standard part of an SEO campaign as more companies streamline their SEO practices. Perhaps it will get renamed as something else?

Sweepsify 

A4: Too many to count! But I’ll start with these. – That you have to have stat sig – That it’s only about A/B testing – That you can use CRO tools – That it’s low value – That it’s too difficult or intimidating – That it’s only for enterprise or programmatic

 Logan Bryant

* It’s highly complicated * The SERPs are too volatile to test * Observational studies are faster/cheaper/easier/more reliable * You need numerous trials to make it concrete * Better things to spend resources on

Darth Autocrat (Lyndon NA)

A5: Absolutely. 1st, start with a simple time-based title tag test. That’s the simplest ‘gateway drug.’ 2nd, there’s time-based content or technical tests. 3rd, bulk testing (A.k.a. multi-URL testing) 4th, A/B Split testing (not to be confused with bulk test)

Logan Bryant

Comparative Splits (A/B) Keep a control sample, alter the test sample, monitor. Multi-comparative (A/B/C…) Same as above, but you alter different things (harder to discern unless you have multiples of each) Reversions (A->A1->A) Measure, change, measure, revert

Darth Autocrat (Lyndon NA)

A6: There are two very interesting schools of thought in data science. Those are “Frequentist” vs “Bayesian” measurement. Fast-forward over the geeky details, Bayesian measurement is the solution to a lack of stat sig.

 Logan Bryant

By breaking away from “proper” statistics, and going with the simple measuring – binary. You want to see if changing X “works”. You change X of 3 pages. You monitor and record. If you see any change in those 3 pages, and not in the majority of others – it’s a 1.

Darth Autocrat (Lyndon NA)

A7: I have to say,

@SearchPilot is doing the sexiest, most advanced SEO testing on planet earth. They have a plethora of case studies with impressive traffic growth. In terms of NON-A/B testing, I haven’t seen other teams reach the scaleability that my Tipalti team did.

 Logan Bryant

Usually, people trying to gain permission for wider changes – such as consolidation or widespread listing changes (schema/titles) Sometimes, the only way to get buy-in is proving the case with a few concrete examples (dummy, low risk, low volume)

Darth Autocrat (Lyndon NA)

A8: Oh boy, I didn’t expect to be plugging 3 times, but I just released a robust CXL course on SEO testing

@CXLdotcom Hard to find a clearer course than this

Logan Bryant

A8: Another shameless plug, but we have some great stuff on the SEOTesting blog aimed at people who are new to the subject as well as more advanced users. There’s also some great content on YouTube!

Ryan Jones

A8 Get them to play with a few dummy pages, on a disposable site (blogger etc.), doing stupid stunts (index/noindex, removing titles, removing internal links etc.). Then get them used to trying to identify what will make the test murky, and how to control it.

Darth Autocrat (Lyndon NA)

The Future of SEO Testing

As the chat comes to an end, the participants contemplate the future of SEO testing. Will it become a standard part of every SEO campaign? Or will it undergo a name change to adapt to evolving practices? The answers are intriguing and pave the way for exciting possibilities in the world of SEO.

In the fast-evolving landscape of SEO, it’s crucial for marketers and website owners to stay ahead of the curve. SEO testing is an integral part of this journey, but it often faces misconceptions and challenges that deter its widespread adoption. By delving into the insights shared by SEO experts in #SEOchat, we can shed light on the significance of SEO testing and how it can benefit your SEO strategy.

One of the primary reasons for hesitancy around SEO testing is the conflation with A/B testing used in conversion rate optimization (CRO). However, it’s essential to understand that SEO testing is a distinct process focused on measuring the impact of changes on search engine performance.

To overcome the perception of complexity, it’s important to view SEO testing as a do-measure-learn process. Start small by making minor changes, such as altering title tags, and monitor the results over a few weeks using tools like Google Search Console. This simple yet effective approach will demystify SEO testing and open the door to valuable insights.

SEO testing is not just a passing trend; it’s an inherent part of learning and improving SEO strategies. As long as SEO exists, testing will remain valuable in understanding the impact of your actions and optimizing your efforts for better rankings and visibility.

While some myths persist, such as the belief that SEO testing requires numerous trials or that the search engine results are too volatile to test, debunking these misconceptions is essential to unlock the true potential of SEO testing.

When considering how to implement SEO testing, several methods are available. Time-based tests for title tags and content provide simple gateways into the world of SEO testing. More advanced techniques like A/B split testing, comparative splits, and reversions offer deeper insights into the effects of changes.

To ensure buy-in from stakeholders, it’s helpful to demonstrate the power of SEO testing with concrete examples and case studies. Presenting well-documented evidence of success will pave the way for broader changes and optimizations.

For those eager to learn more about SEO testing, there are great resources available, such as courses from CXL and informative blogs like SEOTesting. These platforms cater to both beginners and advanced users, making it easier for anyone to dive into the world of SEO testing.

SEO testing is a vital aspect of SEO strategy that should not be overlooked. By embracing this iterative process and leveraging the insights it provides, marketers and website owners can enhance their online presence, rank higher on search engines, and stay ahead in the competitive digital landscape. So, don’t shy away from SEO testing; instead, embrace it as a powerful tool for future success in your SEO endeavors.


Jeremy Rivera

Jeremy Rivera started in SEO in 2007, working at Advanced Access a hosting company for Realtors. He came up from the support department, where people kept asking "How do I rank in Google" and found in the process of answering that question an entire career. He became SEO product manager of Homes.com, went "in-house" at Raven Tools in Nashville in 2013. He then worked at several agencies like Caddis, 2 The Top Design as an SEO manager and then launched a 5 year freelance SEO career. During that time he consulted for large enterprise sites like Smile Direct Club, Dr. Axe, HCA, Logan's Roadhouse and Captain D's while also helping literally hundreds of small business owners get found in search results. He has authored blog posts at Authority Labs, Raven Tools, Wix, Search Engine Land. He has been a speaker at many SEO conferences like Craft Content and been interviewed in numerous SEO focused podcasts.