Get Free Quote

INTERNATIONAL: Measuring Performance When Rankings Are Dead

by Richard Bedford, September 21, 2011

Did we mention rankings are dead?

"Great. Now how do I stop my boss from measuring performance with rankings?"

Each pitch has it's challenges; building a relationship with the new client, presenting case studies, meeting the expectations of the client, agreeing on the price, setting KPIs and outlining the return on investment (ROI) of the anticipated SEO project.

Now, more than ever, justifying any form of marketing investment requires some estimation as to the anticipated return on investment. The predicted ROI and the detailed analysis and methodology behind the numbers is often a key factor in winning a pitch. However, for some clients, somewhere along the way, once the work has been won and the project is underway, rankings become an all-encompassing KPI ─ especially for "vanity" rankings. If you don't manage this early on, the team may lose focus from the original KPIs and true measures of SEO success, such as total non-brand SEO traffic, SEO traffic growth or conversions from SEO traffic.

As SEOs we understand that the traditional concept of rankings as a major KPI is behind us given personalised and behavioural influenced search results ─ an environment where rankings may differ for every user, making them hard to track objectively. However, when you are a consultant there is a requirement to manage expectations with your clients regarding the best metrics to use to measure SEO ROI.

We're not saying that you should not monitor rankings; they are an important signal to monitor search visibility and can help identify any issues or weaknesses with the overall optimisation of a site, but there are some significant weaknesses in using rankings as the be-all and end-all of your key performance indicators.

Where Rankings Fall Short

The weaknesses with rankings include:

unique queries stat
  • You can only monitor a finite set of keywords: It is possible to retrieve ranking data from Google using their custom search API, however it is not possible to monitor every single keyword used to find your website. Google published a statistic claiming that 20 percent to 25 percent of search queries have never been seen before.

  • Rankings are constantly in flux: Excluding algorithmic changes, index updates, client website updates and competitor website changes, different users can see different search engine results pages (SERPs). In other words, it is not an absolute truth to say that a keyword ever ranked at any position without specifying when the ranking was taken, who recorded the ranking, with what browser, tool or device used and from which location the search was made.

    • Universal Search: Do you include rankings from image, video, local results or not? Do you report on these separately?

    • Indented listings: Google may group some results together from the same website to help users. The second listing may not actually rank in that position but would otherwise be ranked somewhere lower down in the SERP. Does the ranking tool that you use report the actual position of the indented result or does it return the visual position? Which one should you report on?

    • Sitelinks: Should you report on Sitelinks?

    • Number of search results per page: With Universal Search, Sitelinks, local search, etc., how many possible search results are there on the page anyway!? An eighth place listing could be "great" one day and "bad" the next.

    • Personalisation: Even if you are not signed in, Google may personalise the search results based on the last 180 days of search activity. Combine this with +1 recommendations from your social network if you are signed in, and the search results between different users can start to look really different.

    • Localisation: Location influenced search results are going to become more important, especially with continued increase in smart phone uptake.

    • When the rankings was taken: Algorithm updates (450+ a year), a constantly changing index (more than 1 trillion unique URLs) and trending searches mean that the rankings may change without any changes made to your or your competitors' websites.

What Metrics Should We Watch and How Do I Convince My Boss?

Very often we do not have all of the data we need to demonstrate the true ROI of an SEO project. Often the SEO project will be running in parallel with an analytics implementation project, a CMS rebuild, a rebrand and a company re-structure, so be prepared to roll with the punches.

  1. Explain the value of the long tail

    • Explain the difference between a head term (e.g. "widgets") and then the long tail derivations (e.g. "buy the best red widget") of the head term.

    • Show them a comparison of the relative search volume for the exact match compared to the broad or phrase match version of the keyword using the Google AdWords Traffic estimator tool.

    • Compare the number of visits from a head term against the cumulative number of visits from the keywords that contain the head term.

    • Show them that head terms may drive traffic, however, more specific and relevant phrases drive conversions and conversions drive dollars, which will give a higher ROI.

    • Longer tail phrases are generally less competitive.

  2. Provide examples of personalised search results; see the original rankings are dead post by Bruce.

  3. Provide examples of the search results from a number of different locations.

  4. Present the average ranking position data in Google Webmaster Tools. See the comments above on rankings being in flux. Where is our keyword or page "most of the time"? Seventy-five percent of the time we are in the top 3 -- awesome!

  5. Look at trends. Are we moving up?

  6. Don't make ranking the focus of the report. Often we are keen to celebrate our SEO victories, however celebrating rankings may lead to further expectations based on rankings. Try to celebrate lifts in traffic or conversions.

  7. Compare SEO to other traffic sources based on metrics that senior managers understand and are comfortable with.

  8. Be patient.

Ultimately this is an awareness and educational exercise. It is often the CEO or senior manager that is driven by the vanity rankings, so sending the right message up the chain of command will be critical. Building future business cases around consistent and measureable metrics such as SEO conversions or total SEO non-brand traffic and presenting these metrics as part of a regular SEO awareness program are the first steps in this often challenging, but rewarding process.


For permission to reprint or reuse any materials, please contact us. To learn more about our authors, please visit the Bruce Clay Authors page. Copyright © 2011 Bruce Clay, Inc.