The Initial Assessment of Google Unveiling Approximate Search Counts in Google External Keyword Tool
===================================================
In this issue of the Keyword Informer…
- Quick Overview of Google’s Release of Search Count Data
- Compare Google Analytics Logs to Google Search Count Numbers
- What Hidden Data on the Google Keyword Tool?
- What does NicheBOT Plan to do About the NEW Google Data?
- Answers to Your Questions
===================================================
1. Quick Overview of Google’s Release of Search Count Data
Ah, the bees are buzzing (much as are the people are in forums)…
Summer is here, and…
Bloggers are blogging on and on about the terrific news that Google has released actual search volume numbers and replaced the old green bar graphs that gave an approximate search volume on Google’s Keyword External Tool.
The switch was impulsive — and it was out of thin air — and very unsuspecting.
I truly believe it caught a LOT of people off guard (especially those in the keyword research community).
In fact, last week, I myself was completely taken by surprise from this unveiling and was shaky to even disclose this fact to NicheBOT’s subscribers because I was just having a hard time comprehending that Google would release their search numbers — just like that!
You see, Google vowed to never allow what happened to Yahoo!’s Overture tool with the numbers being way off (and highly overstated). Something, somehow tipped the cow at Google.
There was all sorts of speculation in the comments to last week’s announcement with questions as to why Google had done this and what their motivation was behind the rather drastic move.
In all severe honesty, I thought the exact search numbers (or approximate numbers I should say) was some sort of glitch — a test to see if someone was paying attention or some belated April Fool’s Day joke. 
Google can show a good sense of humor at times with the copy written on their site.
This type of bare naked disclosure by Google was truly the furthest thing from my mind. In fact, I’ll let you in that I was anticipating another (major) search engine to divulge its numbers and make its keyword data open to the public. Perhaps Google sniffed that out and made the first move to beat them to the punch. Nonetheless…
It’s a huge move for Google and they apparently did it as a result of advertiser feedback that they have acted upon.
I’m gonna have to side with Google on this. If your customers that run your bread and butter advertising system are begging you to know what the exact search counts are to help them out better and make better keyword advertising decisions — how long do you hold back until it starts to negatively impact your business?
It’s the same thing here at NicheBOT. If a number of people ask for the same feature over and over to be integrated into NicheBOT – of course, it’s going to be acted upon (or those people will go elsewhere).
While there was no big hooplah, this was announced to Google Adwords users (which seemed to be the focal point) and, ultimately, the tool’s main purpose is to facilitate and assist their advertisers.
I could also however envision that Google could have easily kept this data behind the Google Adwords center login rather than exposing the data to the Joe Blow public. There was some strict intention to allow the Google search counts to be viewable to the public and basically allow any living, breathing, human being that has access to a personal computer connected to the internet (PDAs and iPhones are not disregarded here) to see the approximate search count of how many people are search for a particular phrase on Google.com.
And just by releasing numbers now immediately brings to mind one immediate question on everyone’s mind.
It’s the same thing most people wonder about other keyword data that comes from other sources . . .
“Is it accurate?”
As I’ve been saying for years and years and in emails to subscribers, my weekly live call-in shows, keyword research is in its infancy at best just yet.
It is NOT an exact science and cannot be taken that way.
The only way keyword research could be even close to scientific is if all the major search engines (Yahoo, Google, MSN, Ask.com and a few others) compiled their information into one major searchable database that online business owners would have access to.
That is not going to happen anytime soon.
However, Google divulging its numbers unveils at least what half the world is searching for — or at least 50% of the world’s population — which is a solid step in the right direction. And to me, this now gives pretty good insight into the collective consciousness (the people around the world).
But I digress.
A good number of people asked (personally and in comments on the blog) — are the Google search volume figures accurate?
And I think a good word of caution arises here.
Just like anything that gets released right away — a piece of software or some neat tool or some major overhaul to a tool, it’s most of the time getting released in beta fashion for the users to battle test it and give feedback while the system is tweaked by the developer (in this case Google).
I think the same kind of attitude needs to be taken with the new found search counts given in Google’s keyword tool which is primarily for Adwords advertisers (but great for keyword researchers). 
In other words, don’t take them too seriously.
As times goes on — I imagine that Google will tweak the system to relay a much more finite, closer estimation as they dial in crunching all the hundreds of millions of searches being done all around the world. We should be grateful just for the fact that they have released ANY numbers to begin with. Really!Google even calls it like it is in their explanation on the site — these numbers are approximations — meaning estimates — and should be taken as such.
What I particularly like better about this data rather than worrying about its accuracy – you can sort out to see what the most popular keywords are — even if they may or may not bring in as much traffic. At least there can be some priority placed on which keywords get promotional attention first.
2. Google Analytics Log Comparisons to Google Search Count Numbers
So just like I promised in an email to NicheBOT subscribers, I am going to share some example comparisons I did so we can see how the Google approximations compare to my own traffic logs that are drawn straight from Google Analytics (my website software tracking package I use for multiple online properties). This is only a small sample taken to really ascertain whether the numbers are anywhere close to accurate or approximations.
I took a good variety of keywords to give a glance of how approximate the numbers are.
Interestingly, Jerry West, as a guest blogger for John Cow, said in his article about Google releasing the numbers, that you’ll need at least 600 impressions for a keyword on your end or there won’t be any data from the previous month (and Google will show “insufficient data” rather than a search count volume).
So let’s see what my search engine logs revealed for traffic and what Google shows for their approximations of searches for that keyword.
I will not be sharing the actual keyword phrases, but rather the position number in the search engine, what the stats for that keyword are and what Google shows for last month’s approximate searches and then the average figure over the past 12 months.
When I ran searches using the Google External Keyword Tool, I ran the search for the keyword phrase first which is displayed in broad search. Then I went ahead and once the results were displayed, I used the pull down menu in the upper right hand corner to choose “Match Type: Exact” because that will show more accurate results.
Here’s what I found:
- A keyword in position #1 brings in an average of 2,628.5 visitors each and every month (over the past 6 months) and Google’s approximate numbers are 1,600 for June with an average of 1,900 searches over 12 months.
- A keyword in position #1 brings in 583 visitors this past month while Google shows that approximately 22,200 searches were done in June with an average of 8,100 searches over the past year.
- Another keyword in the #1 position brings in an average of 8 visitors each month over the past year while the Google tool gives me estimates of 6,600 searches for June with an average of 1,900 searches through a 12 month period.
- Another #1 keyword position brings in 46 visitors over the past month where Google shows 1,000 searches for June and an average of 720.
- A keyword in the #6 spot on Google brings in 202 visitors over the past month, yet Google shows that there were 4,400 searches in June and an average of 2,400 during the past year.
- A keyword phrase showing up #7 on Google brings in 121 monthly visitors, while Google shows an approximate 18,100 searches for June and a 12-month average of 9,900.
As you can see from what Google’s Analytics logs show compared to the search volume generated by the Google keyword tool, the numbers are all over the place and there is nothing exact or accurate about this picture. (NOTE: I took some of the more drastic keyword examples to give you a sort of picture of how predictable the figures can be.)
I urge everyone to take a reality pill here.
When there was AOL data leaked back in August of 2006, it was gleaned from that data that the #1 keyword position on Google gets 42% of the traffic (I’m sure it’s not like that across the board on every keyword phrase — but okay, I’ll run with that).
As you can see from some of the #1 keyword positions up above, there are some that don’t even fall close to the 42% of the search volume figures from the Google External Keyword Tool.
And yeah, there’s a bunch of banter and noise out there as to whether any of this data is at all accurate. Some say that the data is accurate according to their PPC campaigns, some say it’s over inflated and I’m yet to see anyone say they are under-inflated (much like one of my #1 positions above).
Ed Dale made an exciting video about the Google numbers claiming emphatically that they (the Google numbers) are spot on. But when you look closer at the video, Ed does a search for “male yeast infection” I believe, and quotes the search volume number on the screen. Yet, you can see he is quoting “Broad” match — which numbers we know are way overblown.
Then there is Jerry West who says that you must apply the “Exact” match filter to get more accurate results. I’ll have to side with Jerry here.
Now, I’m sure some SEO experts would weigh in on my keyword examples and say “Jim, there must be something wrong with your title tags or you must have really irrelevant keyword phrases for those poor performing top #1 positions.” And nothing could be further from the truth. The keyword phrases are dead on with relevancy of the sites.
And there’s no way you can argue with logs coming direct from Google Analytics.
This is why the best dose of medicine is to take the new numbers with a grain of salt.
Remember what I said above — if anything, these figures now allow you to bring to the top and prioritize going after the more popular keywords. That is at least no more of a mystery with those green bar estimates!
The other things to keep in mind that affect not getting as much traffic as you think you should get for a keyword phrase at the top (or on the first page) of Google:
- Competing Listings – You are competing to get the click with not only 9 other listings, but as many Adwords ads that are displayed along the right side and some more prominent ads above the organic listings.
- The Power and Relevance of Your Own Search Engine Listing – Your site’s search engine listing which is pulled from the title tag of your site (and is the first line of your search engine listing) and the description below the first line (your site’s meta description) have to be persuasive over and above the other sites on the page. Moreover, it has to be extremely relevant to the keyword. If your site listing is not suitable to the web surfers that see it, then you will not get the click and thus, not get the visitor. So it really doesn’t matter how many gazillion searches Google says there are for a keyword.
For some, this is a heavy dose of reality. But the fact is, even if keyword research was scientific (which is still isn’t), there is no way to be guaranteed that you’ll get as much website visitor traffic according to some number you are given from ANY database.
There are just too many factors on the search engine results page — and it’s your job to test which keywords are more effective than others. No one else can tell you what keywords are going to outperform others. And the search engines are certainly not going to hand over stats on which keywords get the highest clickthrough rates for a certain type of site listing.
3. What Hidden Google Data?
Back when I first announced Google releasing the numbers, Frann had commented that:
Something worth looking at is the extra columns not shown by default. I only noticed these when I decided to copy the entire table (using the firefox plugin “copy table”, as the csv didnt give me the data I wanted. When I pasted the table into my spreadsheet, there were all these extra columns…
Frann was in fact right once we downloaded the Firefox plugin that she mentioned. It’s not really hidden data, but there are some extra columns of data you can actually display when you use the “Choose columns to display” and choose “Show all.” This is the extra data you get to see on the page:
So now you also get to see the Esimated Ad Position for a keyword, Estimated Average Cost Per Click, Search Volume Trends over the course of one year and what month the highest volume of search occurred in. And that is some serious extra meaty info that one can use to make some decisions about the keywords, don’t ya think?
4. What does NicheBOT Plan to do About the NEW Google Data and When?
As a market leader, you can believe that NicheBOT is on it!
We’ve already made contact with Google right away to see when they are releasing this data to the API users so NicheBOT members can make use of it. Unfortunately, Google has not given us a definite date.
Alternatively, we already have an import tool that will take the CSVs you can download directly from the tool, and then import the keyword data into NicheBOT for further competitive analysis and so on.
This should be out and available to NicheBOT users by next week.
As soon as this data becomes available to API users, we should have it incorporated and integrated into NicheBOT within hours — not days or weeks.
5. Answers to Your Questions
Last week, when I rang everyone with the good news about Google making this data available, there were a number of questions I promised to answer as well with this follow-up, here are the answers (most of which was covered up above).
Mark Klugman @ 4:51 pm:
This is a major development. Jim, when can we expect to see these numbers reflected on Nichebot’s pages? I am a very happy subscriber. Thanks.
ANSWER: Just like it says above, as SOON as Google makes the data available through their API. Right now, all we get back in response to Google searches are green bars.
Anthony Ettinger @ 5:09 pm:
Is it publicly available via an API from Google yet?
ANSWER: Nope – not yet. But once it’s available, it’ll be up lickity-split.
Brad Hodges @ 5:17 pm:
Hummm…why now? What is Goggle up to? It almost reminds me how a sale goes up to liquidate old inventory before new merchandise comes on board. Google does everything for a reason. While I cannot initially see anything bad about this something is up.
ANSWER: Yup – this was for a reason – and it was due to their advertisers feedback.
Anthony Whyms @ 5:20 pm:
Hi Jim,
Thanks for the head 
About 30 minutes before I saw your mail, I was actually on Google keyword and I thought (to be honest with you) that a bug was affecting my laptop.
Hey, i used this keyword last week and I did not see anything like this – raw data starring me in the face?
Anyway, as I refresh and searched for more keywords on multiple pages I fully realized it was real. However, I have some fears:
1) Has this really come to stay?
2) Can we totally depend/trust this data as the real numbers being search on Google?
3) In some of the search results, you see either under Under eithApprox Search Volume: June Help or Approx Avg Search Volume: “Insufficient Data”
Jim what does this mean?
Will be glad to hear from you.
ANSWER: Hey Anthony, let’s take your questions one at a time.
1.1. I believe this has come to stay.
2.2. Take the numbers with a grain of salt – see above.
3.3. The insufficient data is not defined in Google’s help material, but I take it to mean that there was not enough searches or history or searches for that term and they don’t have any numbers to reflect for that given column, whether it’s average searches over the course of a year or for the previous month. Jerry West had indicated a magic number of 600 visitors on your end for a keyword to see any data on Google’s keyword tool.
OmahaSEO @ 5:40 pm:
This new information begs the question – How close are these estimates? I have already started reviewing recent keyword data and it’s obvious: Google is only providing estimates, not actual data.
If you research a keyword phrase with enough related phrases and synonyms, you will see estimate numbers repeated for different phrases – almost pattern like.
It’s not a straight multiplier to their previous decimal values – but maybe like PageRank, a logarithmic scale.
ANSWER: For me – the estimates are all over the place. See above.
SEO Image @ 6:54 pm:
It does not seem to differentiate between “exact match” and broad match. Since this is a PPC tool, you still need to consider the results may be skewed with “broad” data. Unless I missed that info?
ANSWER: I would definitely use “Exact” match and there should be a differentiation.
Profit Hunter @ 10:54 pm:
Yeah, that’s a really big news!
But has anyone compared numbers which the Google Keyword Tool gives with your real data? I have some very strange results…
ANSWER: Yes – I see some very strange results too and I suggest anyone be leery at first.
Denny – small business software @ 1:53 pm:
I too just did a “test” on 10 main keywords and it does seem largely inflated. My question is WHY would they inflate their own numbers?
Seems like that would discourage using those in an adword campaign, since the green bar usually shows competition at the maximum. The partial green bar or no green bar for comp. and lots of traffic for a popular keyword makes it nearly unbelievable completely.
What are they REALLY up to??
ANSWER: Not sure why they would inflate their numbers. Their database has to be SO huge, that they are probably using some sort of formulation to compute searches across all data centers. Hundreds of millions of searches can quite imaginably be hard to keep up with.
Valheru75 @ 3:07 pm:
This really is amazing!! For the first time ever I don’t have to guess what number of searches keywords are getting on google!! For forever we have all used overtures numbers and a multiplier to get these numbers and we just hoped they were close. This is great but I have to agree it makes me nervous.
Why would Google make this change?
Is it going to stay this way?
People must be scraping the hell out of keyword lists right now to get these numbers into a database.
ANSWER: Google made the change in response to advertiser (customer) feedback. The customer is GOLDEN.
——————————————–
That’s the update to the newly released numbers and hope that makes it clear that I believe your results will vary. You should not get hung up on the fact that a keyword doesn’t produce the amount of traffic the numbers say it should because you’re simply not going to get all the clicks.
If you have any further data you would like to share with other users that might be helpful, please do so below.
Of course, I’d love to hear your comments regardless of having any further data to this update.
Best of success,
Jim Morris, Founder
NicheBOT – “Finds exactly what people search for”
P.S. Spread the word about this update by giving us a Digg or just telling a friend. They’ll thank you for it!