The word “success” is a farsighted and ambiguous term when it comes to the search engine optimization world. Keyword placements are not guaranteed in the SERPs, nor is there an easy way to go about measuring the subsequent conversion value from those keywords (more so after Google’s “not provided” update).

Think about it: After conducting a thorough keyword research analysis of your business, you manage to narrow down your monster list to a very refined set of queries; queries that you spent countless hours on, critiquing relentlessly from all possible angles of opportunity — search volumes, competition levels, SERP topics, business relevance, etc.

Then, your boss fatuously asks you, “Why not choose this keyword instead?” And to complicate matters even more, your digital marketing agency offers its own portfolio of keywords that they believe would be highly profitable to your business.

Now comes the hard part, the decision. Who is right? How would your keyword selection impact your business? Are you willing to throw thousands of dollars and wait for five months (if not more) to go after your keyword selection, praying for the best possible outcome?

Or you can be smart about it and confidently bolster your decision with proof of data.

Let’s consider a more specific example:

Say we’re a secondhand automotive dealer and we have narrowed down our keyword research list to these four queries:

This is a perfect example of a dilemma that any SEOer faces while doing keyword research.

1. The first keyword, "used car dealerships" — “Ah, that search volume looks so juicy! We’re not doing terribly bad. Only if we can push forward a few more paces, we’d be swarmed with web traffic. However, with great search volume comes even fiercer competition.”
2. The next one, "second-hand cars"— “Oh boy! We’re already on the second page and the competition level seems doable. Hmm, but the search volume is not nearly comparable to the first one.”
3. Third, "cheap cars for sale"— “This is another interesting one. But we’re not sure of the intent of the searcher. Are they looking for cheap new cars or used ones?! How would they react if they came to our website?”
4. "Used SUV for sale" — “Look! These people practically have their whole wallet out. They seem to be deep inside the sales funnel, looking for a specific car brand. This keyword could have a lot of potential for us. But the low search volume, along with the current ranking on this one, is both intimidating and heartbreaking. Not sure if this keyword would really be worth all that effort.”

What’s the solution? Let’s jump right to it.

Experimenting with pay-per-click (PPC) ads

Yes, you can leverage paid ads to choose which keywords to target for your organic search strategy. The motive behind this is to get web traffic samples from each query, monitor each group’s behavior using performance indicators and finally compare them against each other to know the winner. This can potentially save you loads of time and money by avoiding the catastrophe of channeling all your efforts in something, only to find out later that it’s as worthless as 97% of your web traffic.

A few points for implementing the experiment

1. To set up the experiment, we can create a campaign with multiple ad groups under it, each of which would show an ad for the keywords that we have targeted.
2. While typing in the keywords in our AdWords ad groups, we’d want to make sure they are all set to "exact match" because we wouldn’t want our ads to show up for queries such as cheap car toys for sale... :)
3. To replicate this experiment as closely as possible with the top organic listings, we want to set the bid strategy to "target search page location" to have a higher chance of coming up above the fold among the first few results.
4. The ad title and description should also be written as if we’re writing text for the title tag and meta description for the page.
5. Before doing the experiment, make sure conversion/event tracking is set up correctly and Google Analytics is linked to your AdWords account.
6. Finally, set your daily budget, bidding strategy and delivery method. Depending on your business and search volume of keywords, you can set your ad delivery method to ‘standard’ or ‘accelerated’, meaning how quickly your ad would be shown each day. We have decided to run our experiment for eight days on accelerated ad delivery.

More advanced users can set up the experiment using the AdWords experiment tool.

The results

We know the average CTR for the #1 position in the organic search results is estimated to be somewhere between 18% and 36%. Using Google Search Console, we can estimate the CTR of these keywords based on similar queries that are ranking high. After looking at our Google Search Console data, we have found our CTR to be approximately 22% for informative queries and 20% for transactional queries.

Using this data, we can now predict our organic traffic and conversions for all these keywords as follows:

We can see that the "cheap cars for sale" keyword would yield us the highest conversions due to the high search volume even though "used SUV for sale" seems to have a better conversion rate.

We have all our experiment results. But the question now is how accurate would these keyword performance metrics really be if we decided to go after them on a much larger scale on the organic side of things? Results from experiments like these are often subject to what is called sampling error, random noise, fluctuations, chance, etc. After all, we only have a small sample subset from such a vast pool of searches. Now the challenge is to separate the noise from the signal to get a more reliable and valid prediction.

But first, I must warn you: From this point onward, it's just math.

If you just happen to be involved directly (or indirectly) in analytics, these tests might give you some nasty memories from school.

Don’t worry, I have included a spreadsheet that will compute all these calculations for you. All you need to do is feed it data. So, feel free to skip the next part and go straight to the statistical model if you feel uncomfortable with the math.

We will compare our sample conversion rate to a benchmark conversion rate to predict the probability that it would exceed the benchmark if we decided to move forward with a particular keyword for SEO.

For the nerds: We can do this using the Z-Test for binomial proportions. This test is eligible if we have a sample size that is large enough. There is no magic number to determine this, but a rule of thumb is a minimum of five successes and five failures. The formula for this is:

z=(p1-p)/sqrt(p(1-p)n)

P1 = observed conversion rate, p = benchmark conversion rate, and n = sample size (number of visits or clicks).

So, for example, if we wanted to compare the sample conversion rate of "cheap cars for sale" against a benchmark of 1%, we would plug the values from the experiment in the spreadsheet calculator, which would give us a confidence level of 89%. This means that about 9 out of 10 times, the conversion rate of "cheap cars for sale: would exceed 1%. And we know a 1% conversion rate for this term equals 99 conversions (0.01 * 9900).

We can work this in reverse to find the benchmark rate for a 95% confidence interval using the "goal seek" tool in Excel. Just download the spreadsheet, open the "goal seek’"tool and type in these values exactly as they are.

After clicking "OK," we get the following result for the "cheap cars for sale" keyword:

As you can see, the confidence level is very close to 95%. Thus, based on this statistical analysis, we can predict 95% of the time that our keyword "cheap cars for sale" would exceed a conversion rate of 0.86% (or 85 conversions). This is a reasonably high probability with only 1 in 20 times that it could be wrong. Now if we know the average revenue per conversion, we can estimate revenues from keywords, too! Assuming our average revenue per keyword is $1,500, this means 95% of the time we can expect to earn at least $127,500 (85 * $1,500) from the "cheap cars for sale" keyword with a 20% CTR and search volume of 49,500 per month.

Now, for bonus points we can also compute a probability based conversion range that would predict with 95% accuracy where the actual values would lie if we got organic web traffic from all four keywords.

We will use the adjusted Wald confidence interval formula for binomial proportions as it works better for both small and large sample sizes.

C.I=p(adj)+- z * sqrt (p(adj)(1-p(adj))/n(adj))

Where p(adj)=(x+(z^2)/2)(n+z^2)

Plugging in the values in our spreadsheet calculator, we get the following output for all 4 keywords that we had experimented on:

So, 95% of the time the actual conversion rate of our 3rd keyword "cheap cars for sale" would lie between the range of 0.7% - 3.29%. This means if we decided to go with the keyword "cheap cars for sale," we would get somewhere between 69 and 326 organic conversions with a 20% average CTR. From the revenue calculations above, we can expect to earn somewhere between $103,500 and $489,000 per month.

Fascinating, right?

Two key points to remember when doing experiments like these:

1. Never skip the statistical analysis — It would just make your predictions unreliable, inaccurate, and invalid altogether.
2. Aim for a 95% confidence level, even though you can go for other confidence levels depending on the context.

“Without data you’re just another person with an opinion.”
-W. Edwards Deming

Now, please take a moment to comment below on how you go about predicting conversions for the organic channels you manage. Let us know if you agree/disagree with the analysis in this post and share any suggestions you may have for improving the process.