Are rankings a reliable metric for SEO success

by Peter Young on March 23, 2010 · 16 comments

Times are a changing. I started SEO back in the good old days, when SEO had an air of mystique about it and unfortunately the perception as a bit of a ‘dark art’. This perception is still something that you find today, albeit I would add that this is the exception rather than the norm.

Much of this can be attributed could potentially be attributed to a number of factors however communication, lack of accountability and general terminology all probably played a part in that perception. These days however, marketing managers are becoming more and more online savvy and more aware of both what SEO is and what it involves.

Further to this, Google and the other search engines are a work in progress. The Google of 5 years ago may look very similar, however behind the scenes is a very different picture. The advent of personalisation, the rise of social media and the development of frameworks able to deal with real time indexation and evaluation means that Google is able to return far more dynamic results than ever before and as a result the metrics used to evaluate are having to move with this.

In particular personalised search could potentially have the biggest impact on measuring SEO success. Working in the industry it still amazes me how many times you come across organisations who still measure the success of their campaigns onhe position of their sites within the search engine results pages, however this approach carries a number of caveats

  1. Personalised search means that whats number 2 for me, may not necessarily be number two for you. Based on my behavioural trends number two could be a far different scenario – as could many of the results below.
  2. Results are not constant. Whats here today may not be there tomorrow. The search engine results are hugely dynamic, something I believe is only likely to increase once Caffeine is fully rolled out across the Google framework. Thus content won’t have as much of a lag effect, potentially indexed, evaluated and returned within a matter of minutes. Old school results were a far more reliable and consistent metric
  3. Rankings does not necessarily mean traffic. OK that comment certainly needs further breakdown as being number one for a ‘jackpot term’ such as car insurance certainly is going to mean traffic, but the amount of examples I have seen which highlighted ranking wins on obscure terms which are never likely to produce more than a handful of visitors. Further to that ranking on vanity terms may produce traffic, but if its not relevant to conversion that visibility is flawed.

As such I believe that rankings are a KPI living on borrowed time. Yes, we can look at results by turning personalisation off however how many users actually do that when they search. Therefore success needs to be based on very different metrics, and SEO campaigns have to bear this in mind as this not only affects how we measure SEO, but the framework and strategic focus of the campaign from the very outset.

In particular incremental sales, should be considered an absolute essential aspect of modern day SEO campaigns. With more and more tools allowing search marketeers (both paid and organic) to measure their campaign success both in terms as a standalone channel and as part of the integrated mix, the requirement to put more solid and commerically focus KPI’s being campaign success is now a no brainer.

Further to that, modern day search campaigns are far more widespread than simply optimising websites, which once would have worked. Social media, PR, Brand Advocacy amongst others have all eaked their way into modern day SEO campaigns, and thus the impact of work undertaken by us SEO’s may also be extended to include brand noise and coverage.

SEO has grown up, and with that maturity needs to come accountability. Measurements by rankings may be obselete but in its place are far more tangible measurements of success, and ones that could have a far more significant effect on our bottom lines in the long run…..

Google+ Comments

{ 15 comments… read them below or add one }

Brett Pringle March 24, 2010 at 7:45 am

Hey Pete,

While you do seem to mention that ONLY relying on rankings as a measurement for success has seen it’s day, it does however still play a role. Whether it is the “jackpot term” or the niche longtail that provides those small number of visits, done correctly they play a large role in the performance of the site, not just from a traffic perspective, but also the conversion point of view as well. That “jackpot term” could easily provide large amounts of qualified leads/sales to a website (if done correctly)

Rankings flux daily if not hourly these days compared to the past, however using the average rankings for a site in conjunction with other analytics metrics allows marketers to gather data and make more informed decisions on the next step for campaign. Running reports using third party apps to track rankings is just not sufficient these days, although a quick analytics filter addition can provide a large amount of insight into website rankings, traffic recieved from the various positions and which search phrases and positions convert the best.

Personalised search may play a role, however i doubt very much in the scale we think, especially after the tests completed from the SEOdojo peeps awhile back. Very possibly the reason Google changed the customised results to the “bookmarking” style with the star feature seen when signed in recently.

“Rankings” will always play a large part of search, whether that be blended search integration and ensuring map results for search queries through to social media within the SERPS.

Let’s hope the old school ranking measurement done by companies still today dies out, as website owners become more educated and start to focus on ROI. That should definitely the key measurement for the success of any marketing online.

Adapt to changes or lose out?

Peter Young March 24, 2010 at 9:28 am

Hi Brett

Thanks for the comments, completely agree with your points above

I would add that much of the article above is aimed at organisations that still continue to use rankings as their primary KPI with clients, something which as you have pointed out is not a constant and something which is often in a state of flux.

However sales, leads etc – those are tangible, they are measurable and ultimately I would argue this is what most clients will be interested in, not whether they are number 1 for ‘metal magnet blackburn’.

jaamit March 24, 2010 at 10:32 am

Pete – agree with your points above – especially given Google has said a massive 1 in 5 results are either personalised or localised (quite insane stat that!).

But I’m also with Brett: rankings should not be thrown out as a metric. As SEOs we are in the business of gradually increasing the authority and relevance of pages on a site for particular keyphrases, and tracking change in (non-personalised) rankings compared to a baseline will always be the primary metric to measuring whether our efforts are working or not.

But as far as KPIs are concerned you’re absolutely right – showing “we’ve achieved position x for this keyword” or even worse “we’ve achieved x page 1 rankings” as evidence for SEO success is pretty crap and is frequently used as an excuse for bad SEO. I always try and take it as far down the conversion path as possible – traffic, conversion events, leads, sales/revenue and even profit margins if possible. SEO’s are really slack on providing ROI stats on their work and this needs to change if we’re going to survive when the recession really hits our industry.

Peter Young March 24, 2010 at 10:52 am

Great feedback Jaamit .

Certainly rankings from a practitioner perspective are still alive and kicking, you can’t proactively develop a campaign without knowing where you stand first – however as all parties appear to agree as a KPI metric for accountability purposes a ROI based model is a far better metric of overall success for client purposes

Ryan Beale March 24, 2010 at 11:52 am

Hi Pete,

Great article. While ranking on Goggle’s Organic SERPS is important, I believe that a few things that will always hold true to continually bring high quality traffic and leads/sales are 1) continually creating well optimized, fresh content (which will increase the chances of high quality websites linking to your content).2) Having a social media presence will continue to be important with google’s real-time search results includes relevant conversations (caffeine).

My $.02

Usman Patel March 24, 2010 at 12:12 pm

Hi Pete,

I work on commercial sites for SEO so timely article.

I do think that visibility via search (rankings) is something we do need monitor. And I agree, rankings certainly does not mean traffic, especially if you use Google Webmaster Tools, you may get the impressions but not necessarily the click-throughs!

But if you optimise your site for the long tail i.e. site architecture, this is where bulk of the traffic will come from not forgetting the short tail!

However I’m firm believer on conversation rate optimisation and creating pages to ensure conversion is crucial i.e. price points, nice imagery, CTA (something Jaamit mentioned in his feedback).

Also the most important metric is, driving those new visitors in which the first point of contact to the site is crucial and more so for brand visibility. So measuring new vs. repeat visits is something you need to monitor. So ranking has some relevance. I tend to find rankings (non-navigational) doesn’t always correlate to sales, I know because I’m just analysing the stats for a site I’m working on. But to caveat that, branded terms convert the best, I suppose coming to the site initially via entering a keyword.

Great feedback by the way from Brett and Jaamit.


Ryan Blakemore March 24, 2010 at 1:06 pm

Hi Pete,

It’s great to see people looking past the blanket ranking change model that we have been using in the past.

I think you hit the nail on the head with the point that ROI should be what we are looking at. The only problem with that is that an ROI figure is client friendly, but isn’t as accountable as rankings and it isn’t the most helpful number for SEOs to be working with. I think internally we need to be looking at the same metrics as we present to our clients, otherwise we are playing a dangerous game hoping that the causation fits with the correlation.

I think the best way to look at it is more of a fusion of the two. We need to extract keyterms and assign values and accountability to them. If we can add context to the keyterms, we can ensure we are working towards the ROI numbers but we are still able to track our progress and set targets effectively.

Peter Young March 24, 2010 at 1:12 pm

Re: the ROI figure I would have to say that times are a changing. Certainly in the past, analytics tools havent allowed for precise measurement however these days there are more and more tools coming to the market which allow for greater and more precise measurement of organic search accountability

For example, I understnad Doubleclick are bring a natural search product to the market , which will allow for far greater transparency and analysis for us SEO practitioners.

Jeff Swanson March 24, 2010 at 2:55 pm

I agree that rankings are not reliable because of all the points mentioned above. That doesn’t mean it’s not useful to see where you’re ranked, to get an idea of what to work on; but just wouldn’t rely on them in reports to clients or other people in your company.

George March 25, 2010 at 8:55 am

It is not ranking, but it is the click through rate that matters. For that you have to have a tempting and well written content which shows up there as snippets in the search engine results page..

Peter Young March 25, 2010 at 9:30 am

Hi George. I am sorry but I have to completely disagree with your comment for a number of reasons.
1) CTR means nothing from a conversion perspective. If I get a click through rate of 100% and a conversion rate of 0% its not going to make me any money, and thus is completely useless
2) Snippets aren’t something you fully control. Try a 3+ phrase search on Google and you will notice that G tries to return contextually relevant content from the copy as part of the page.
3) How do you measure organic CTR? If you find a tool that can do this well (other than taking PPC exact match data and amalgamating this with your organic), then please tell me.

Zack March 25, 2010 at 1:34 pm

Yep, good post. Rankings can be a reliable metric as long as it goes hand in hand with your KPI. If you sitting nice for a “money term” then I think it is a valuable metric. But it is only a relevant metric if the keyword converts.

Misscj March 28, 2010 at 7:58 am

I completely agree with you Peter.

Krumpet April 15, 2010 at 11:22 pm

Rankings don’t mean traffic? Are you insane? For you to cite high rankings for terms nobody is clicking on is silly. If you are optimizing without figuring out what people are likely to click on then you are clueless to begin with.

The job of an SEO is get traffic through search engines by optimizing content to move into the best positions to get clicked on through better bot analysis and link acquisition. How can you possibly argue against that position being #1? In which case, rankings DO matter.

Don’t even start talking about conversion rates because that is outside the scope of pure SEO. SEO is about getting the traffic. If you happen to be an SEO who also works on optimizing your site to improve conversion rates, great! That is fun stuff, but it IS NOT pure SEO.

This argument (and other similar arguments such as whether or not to provide ranking reports) boil down to the agreement with the client and what the goals of the engagement are. If the goal is to increase traffic then you better improve rankings or cranking out more landing pages. If the goal is to improve conversions, then you better start testing new designs.

Peter Young April 15, 2010 at 11:40 pm

Thanks Krumpet for your comments, I will answer your above comment in parts
1) Rankings != traffic. Obviously we are not talking across the board. With reference to the post above we are talking the 1500 number one terms for client x of which 80%+ never produce a click. It does happen – and by commercial SEO’s – still… I fully agree with your points regarding what incorporates a good SEO – however the fact is there are still people out there peddling that rubbish.
2) Re What constitutes best KPI of success is down to the client, however as the post explored – one has to query whehter this should solely come down to ranking.

Traffic is certainly one to reference by, but if other channels such as Paid Search are judged by their commercial success then one has to ask – why not SEO?

Perhaps you may have a suggestion – given your statement in the third paragraph?

Leave a Comment

{ 1 trackback }

Previous post:

Next post: