is-ctr-a-ranking-factor

blog

Click-through Rate is Not a Ranking Factor – A Case Study

This post is a part of my SearchEngineLand publication – Is CTR (Click-through Rate) a ranking factor? I gathered all the technical data here for readers that would like to go a little bit deeper, but I strongly encourage you to read the article on SearchEngineLand first.

The idea behind the experiment

If CTR is a ranking factor indeed, then manipulating it will also influence a website’s position in Google rankings.

Simply said: High CTR – Higher position in Google

Manipulating the Click-through Rate for my domain

What could be the ultimate test? Sending traffic to a domain that will look like a viral trend (like TV ads, product launch, etc.). If CTR is really an important ranking factor, then obviously this test should result in a major increase in organic rankings.

Increasing the click-through rate

This is the most important part of this case study, obviously. Figuring out the proper way to do it took me quite a lot of time.

Click-through rate increase method

Google is excellent with filtering fake traffic. There are many tools for increasing CTR on the market right now. None of them did the trick for me, though. My goal was not only to send the traffic that won’t be filtered out by Google; I also wanted a ‘black and white’ proof that Google didn’t filter this traffic out.

My plan was to

  • Increase the organic traffic (measured by Google Analytics and Google Search Console).
  • Create a (fake) Google Trend.
  • Manipulate Google Traffic data (that we can see in Google Keyword Planner).

What is the value of this data?

There was no CTR case study yet that could be based on unbiased data. Most of the cases I read so far were crowdsourced. There were quite a few ranking factors and metrics that could interfere with the data. I really wanted to focus on extracting CTR from all the other ranking factors. For example, Rand Fishkin carried out an interesting experiment in which he managed to increase the position of his website from #4 to #1.

Have 20 seconds? Would love help testing this. Just follow quick directions in the graphic: pic.twitter.com/oSf7lAkyma
— Rand Fishkin (@randfish) June 21, 2015

Unfortunately, the results quickly reversed to the same state it was before the experiment.

If you compare this result to Rand’s CTR test from 2014, you can clearly see that CTR doesn’t have as much influence on rankings as it once had.

CTR is not a ranking factor

After conducting this experiment, I strongly believe that CTR is not a ranking factor in 2015 anymore. I firmly believe that the user experience plays an enormous role in the Google algorithm. Still there is more to the user experience than click-through rate only. There are backlinks, content, mentions, social media, plugin data, Chrome data and many more metrics that we may not be aware of. That is why I created one more experiment with three separate domains with no backlinks, no “natural” traffic, etc. I will get to this part a little bit later. Now let me guide you through the bot development process.

Bot development

As I mentioned before, I tested almost all of the CTR bots on the market, and none of them were good enough to deliver the desired results on a scale that I expected. I ended up creating my own bot + setting up a network of servers getting Google proxy. This way I was able to send up to ~30k successful visits daily – visits that were not filtered by the Google algorithms (therefore visible in Google Analytics and Search Console). Getting real user IPs (proxies) was the most difficult part and I had two dedicated, powerful servers doing just that.

Number of clicks in Google Search Console

After setting everything up, I started to send queries to Google for the keywords mentioned earlier.

Structure of each query:

  1. Open browser,
  2. Go to Google.com,
  3. Enter the query and click on “Google Search”,
  4. Search for my domain in the search results page. If not found, go to the 2nd page and so on up to the 12th page,
  5. Click on the result,
  6. Stay on the website for ~2 – 4 minutes while going through random pages.

This is how the traffic was visible in Google Analytics.

Real Time traffic from Google Analytics

Search Console. Clicks, impressions, and CTR

Experiment

Benchmark

Any significant trend has to be precisely measured. In this case I will use a few different factors to make sure that this experiment was done properly.

  1. Google Trends – if a lot of people are searching for something, it creates a clear and measurable trend.
  2. Number of searches in Google Keyword Planner.
  3. Organic traffic (in Google Analytics and Google Search Console)
  4. Increased Click-through Rate (in Google Search Console)

Domain

https://goralewicz.co

Keywords used

  1. Goralewicz
  2. Goralewicz.co
  3. Goralewicz.co negative SEO
  4. Negative SEO
  5. International SEO consultant
  6. SEO Consultant
  7. SEO training

Overview of the traffic in Google Search Console:

Search Console for Goralewicz.co – overview

Traffic sent (keyword by keyword)

Goralewicz

Traffic in Search Console

Search Console – keyword

Click-through rate in the Search Console

Search Console – keyword

Average position in Search Console

Search Console keyword “goralewicz” avg.position

Google Trends

http://www.google.com/trends/explore#q=goralewicz&cmpt=q&tz=

 

Google trends keyword “goralewicz” interest over time

Google AdWords Keyword Planner

Google AdWords Keyword Planner keyword “goralewicz”

I chose this keyword only to increase my CTR throughout many different keywords. As predicted, the rankings remained unchanged here:

Search Analytics Avg.position keyword “goralewicz”

Goralewicz.co

Traffic, CTR and Average position in Search Console

Search Console keyword “goralewicz” clicks, impressions, CTR and position

Google Trends

http://www.google.com/trends/explore#q=goralewicz.co&cmpt=q&tz=

Google Trends keyword “goralewicz”

Google AdWords Keyword Planner

Google AdWords Keyword Planner keyword “goralewicz” may 2015

This was also a branded keyword. As you can see there is no drop in Search Console as well.

Search Console keyword “goralewicz” position

Goralewicz.co Negative SEO

Traffic in Search Console

Search Analytics total clicks keyword “goralewicz” and “negative SEO”

Click-through Rate in the Search Console

Search Console keyword “goralewicz” and “negative SEO” Avg. CTR

What is interesting, with this keyword I managed to achieve high CTR, due to the fact that my website always showed up as the top 1 (better success rate with proxies).

Search Console keyword “goralewicz” position.

Search Console keyword “goralewicz” and “negative seo” total clicks smd total impressions

Average position in Search Console

Search Console average position keyword “goralewicz” and “negative seo”

http://www.google.com/trends/explore#q=goralewicz.co%20negative%20seo 

Google Trends march 2015

Google AdWords Keyword Planner

Google AdWords Keyword Planner keyword “goralewicz” and “negative seo”

Results

Search Analytics avg.position keyword “goralewicz” and “negative seo”

International SEO consultant

I chose the “international SEO consultant” keyword as it seems fairly easy to rank for and I was already ranking for this keyword at around position #30 – #40 for some time.

Search Analytics avg. position keyword “international seo consultant”

If CTR is really a major ranking factor, we should see a significant ranking increase here.

Traffic in the Search Console

Search Analytics total clicks keyword “international seo consultant”

Clickthrough Rate in the Search Console

Average CTR was 41.84% for an average position of 37 (!).

Search Analytics Avg. position and Avg. CTR keyword “international seo consultant”

Average position in Search Console

Search Analytics avg. position and total clicks keyword “international seo consultant”

Google Trends

http://www.google.com/trends/explore#q=international%20seo%20consultant

Google Trends keyword “international seo consultant”

Google AdWords Keyword Planner

Before

Google AdWords Keyword Planner Before

Google AdWords Keyword Planner During

Google AdWords Keyword Planner After

Influence of the increased click-through rate on the average position.

Search Analytics avg. position

As you can see in the screenshot above, the average position not only did not increase, but actually decreased slightly over the following weeks.

Search Analytics avg. position keyword “international seo consultant”

My website never ranked for this keyword (and it still doesn’t). I wanted to test 2 hypotheses with this test:

  1. Massive searches with no clicks (users didn’t find what they were looking for) can cause ranking fluctuations.
  2. Increased organic traffic for “international SEO consultant” will improve my rankings for “SEO consultant” keyword in a similar way in which Google looks at link anchors.

Traffic in the Search Console

Search Analytics total clicks keyword “seo consultant”

Click-through rate in the Search Console

Search Analytics CTR keyword “seo consultant”

Average position in the Search Console

Let’s take a look at the most important part – click-through rate and average position.

Search Analytics avg. position and CTR keyword “seo consultant”

The data in the screenshot above is a little biased. Due to technical problems (it was hard for the bots to find a website on a position higher than 100), the bots only managed to click on the website when it was presented in the top 100. Of course, rankings for each query of my website were a little different. This resulted in my website being clicked and presented more often when it was presented higher in SERPs, and hence lowering my average position.

Google Trends

https://www.google.com/trends/explore#q=seo%20consultant

Google Trends keyword “seo consultant”Google Trends keyword “seo consultant”

Google AdWords Keyword Planner

Google AdWords Keyword Planner keyword “seo consultant”

Google AdWords Keyword Planner keyword “seo consultant” March

Google AdWords Keyword Planner keyword “seo consultant” April

I increased the number of monthly searches in Google Keyword planner from the average of 16,000 to 20,000 visits monthly, to 382,000 visits monthly.

“SEO consultant” proved the hardest for me to get any traffic, as my website wasn’t ranking anywhere in the top 150. Even though for a small % of queries, Google presented my website around the 70th position. Still I managed to get 7,686 visits, which I think is a huge success.

Results

Search Analytics. avg position March-Jun

Over the weeks following the experiment, the average position remained unchanged – fluctuating between ~200 and ~130 until today.

Negative SEO

I had my blog post ranking for this keyword between #2 and #4 for a few weeks already.

Keyword “negative seo”Keyword “negative seo”

Traffic in the Search Console

Search Analytics total clicks keyword “negative seo”

Click-through Rate in the Search Console

Search Analytics CTR keyword “negative seo”

Average position in the Search Console

Search Analytics avg position keyword “negative seo”

Search Analytics avg position keyword”negative seo”

Search Console – combined view

Search Analytics position total clicks impressions CTR keyword “negative seo”

Google Trends

Interesting email from Google:

Google Trends keyword “negative seo”Google Trends keyword “negative seo”

https://www.google.com/trends/explore#q=negative%20seo

Keyword “negative seo”

Google AdWords Keyword Planner

Google AdWords Keyword Planner keyword”negative seo”

SearchMetrics – rankings before the experiment

SearchMetrics rankings before the experiment

SEMrush – rankings before the experiment

February

SEMrush rankings before the experiment

March

SEMrush – rankings March

SEMrush – rankings April

SEMrush – rankings May

Results

After the experiment, the positions remained flat for two months only to start plummeting around June 1st 2015.

SEO training

Traffic and CTR in the Search Console

Google trends

http://www.google.com/trends/explore#q=seo%20training&cmpt=q&tz=

Google trends interest over time keyword “seo training”

Google AdWords Keyword Planner

Google AdWords Keyword Planner May-April

My position for the keyword for “SEO training” was always between “not found in top 100” and page 10 in Google.com.

January

February

March

April

As you can see in January, there were only ten days with any position recorded, then the number dropped to 4 in the next two months to eventually get down to 0 in April.

Now let’s take a look at the data from the Search Console

Results

CTR vs. Average position

CTR vs. Average positionCTR vs. Average position

As you can see from the screenshot above, the significantly increased click-through rate didn’t improve my website’s average rankings for this keyword.

Bonus experiment

I did try this experiment on a few other domains to make sure my results were right. One more experiment that is worth mentioning was done using weebly.com dummy subdomains.

Ctrbot1.weebly.com

Ctrbot2.weebly.com

Ctrbot3.weebly.com

I filled them with really similar (but unique) content and left those domains alone for ~7 – 8 months.

Methodology

As you can expect, I wanted to check if sending fake traffic to 1 of those 3 domains would impact their rankings. All I had to do now is pick one of three subdomains to send fake traffic to.

I went for the subdomain that was ranking lowest to make sure we could see the ranking jump clearly. My choice was ctrbot3.weebly.com.

Sending traffic to one of the domains

I had Google Analytics & Google Webmaster Tools accounts set up for all three websites from day one. This way I could clearly see the amount of Google organic traffic sent to the domain.

As you can see there was a massive Google organic traffic sent to ctrbot3.weebly.com between March and April 2015.

Now let’s compare the traffic with the other two domains.

I sent around 190,000 Google organic visits to ctrbot3.weebly.com.

All the visits you can see above were sent using only a few keywords.

Keywords:

ctrbot3.weebly.com

Google CTR bot

ctr bot

ctrbot weebly

Experiment environment

All three subdomains had the same number of pages, no backlinks, tweets etc. This created an isolated (as much as it is possible) environment to test CTR influence.

Google Trends & Keyword Planner

Let’s have a look at the combined data for all the keywords targeted in this campaign (exact match)

As you can see above, monthly searches jumped from 0 to almost 1 million. Of course this data is only for the keywords used in this experiment (see screenshot below).

Google trends also noticed this unusual traffic spike for the keywords used in my experiment:

https://www.google.com/trends/explore#q=google%20ctr%20bot

You can see similar data for all the keywords used in this experiment.

Control group

To make sure that the methodology behind this experiment was good, I was tracking all three websites closely to make sure that none of my bots were hitting the proper landing pages and to make sure that there was no other traffic that could influence the experiment results.

Results

This experiment was unfortunately stopped by weebly.com. All three dummy websites were removed (most probably due to heavy incoming traffic). Because of this, I lost the Google Search Console access with the exact CTR data. Fortunately enough, before those got removed, I was able to see the results of the test.

March

April

As you can see in the screenshots above, rankings didn’t go up until they plummeted completely when the Weebly page was removed.

No ranking improvements even for long tail keywords

All the keywords used in these experiments were easy to rank. In fact, as you can see, the dummy weebly.com page was ranking pretty well for all those keywords with no links, optimization and filled with just one page of content. I think this example proves that CTR is not as powerful as it was before. In fact, after looking at the data from all my experiments, I believe that CTR has no direct influence on rankings whatsoever.

Two different CTRs

During this experiment, I managed to get some data to support my hypothesis that there are in fact two different click-through rates for each single query.

My hypothesis:

  • One that can be found in your Search Console based on a simple algorithm.
    Your website was presented 100 times in Search Results, and it received 10 clicks. Therefore, your CTR is 10%.
  • General database with the number of queries and number of clicks for each website.

Interesting fact:

If your website gets ten clicks and the CTR of your website is 10%, it doesn’t mean that other websites get 90 clicks (from the total of 100). You can get 10% of the queries even when 90% of the queries remain unanswered (not clicked at all). It is good to know when explaining analytics to your clients and going over the results you provide for your clientele.

Outro

The results of this experiment were surprising even to me. I was the only person working on this experiment, sorting out proxies, servers, and the whole setup. Still I managed to fake not only Google trends, but also the very core of any SEO work – traffic and AdWords value in Google AdWords Keyword Planner.

Funnily enough, Google Trends are used by a lot of authorities to predict stock market trends or, what is quite unusual, a presidential election!

Still the whole experiment only proved what I firmly believe in. SEO is not about finding a loophole or a trick; it is hard work based on a lot of knowledge and experience.

SEO is hard work. Google has their hands full maintaining the user experience at a level which allows them to continue making the general population happy. With all of us working and doing our best with the resources we have, it is important not to jump on the latest fad. Just like eating chocolate for weight loss was once a trend that picked up momentum, we need to grow as a community and clear the facade.

With so much information out there and people who spin a lot of smoke and mirrors, it is important for us as a community of experts to rely on hard facts and technical data to deliver the best results for our clients. It is also critical for us to prove and disprove speculations by cleaning up our practices, offering transparency and replicable results to give clarity to our former, current and new clients.

Image Credits:

https://commons.wikimedia.org/wiki/File:Mythbusters_San_Diego_ComicCon_2009.jpg

Published
  • 12 August 2015
Comments
Category
Bartosz-Góralewicz

See all articles by Bartosz Góralewicz

Did you like this article?

Why not share this article:

Be in the loop. Get fresh SEO and Content Marketing updates!

Thanks! We are happy to have you on our list!

Expect some tasty news from the Elephate team soon.

more blog posts

Back to Blog list