Search Quality Rater Guidelines Checklist: Evaluator Considerations
EDITOR’S NOTE: You can always find Google’s current Search Quality Rater Guidelines here. The latest update was released December 15, 2022.
Google’s update of its Search Quality Rater Guidelines shows a shifted focus at the search engine, and consequently, for SEOs. BTW, the Google PDF file name says Evaluator and not Rater… but it is Rater.
For years, the buzzword for search engine optimization has been “relevance” — making your site the most relevant result for a searcher’s query. But as Duane Forrester, our former VP of organic search operations, observed: “It’s all about usefulness today and moving forward. The goal of the search engine is simple: increase searcher satisfaction. That’s why ‘useful’ is the new watchword. Previously we said ‘relevant,’ but really we all meant ‘useful.’”
Google regularly updates its internal guidelines document that tells hired human quality raters how to evaluate sites as part of Google’s ongoing experiments. We in the search industry usually get only leaked tidbits and summaries to read. But last month, in a rare gesture, Google published the guidelines as a PDF for all to read.
While it doesn’t reveal any ranking formulas or algo secrets, the 175-page document complete with many examples and screenshots does offer a coveted view of what the search engine considers priority. As Google’s announcement states, “the guidelines reflect what Google thinks search users want” and therefore can help webmasters and business owners “understand what Google looks for in web pages.”
The guidelines are not the algorithm. But they do show what Google’s focusing on, and that’s worth paying attention to.
What’s important for business owners is not all of the nitty gritty technical details. Leave those to your SEO. Instead, business decision makers need to glean what Google’s focus is so they can allot budgets and assign priorities correctly in a website strategy that’s aligned with what Google rewards.
Aligning Your Website with Google’s Priorities
Search engine priorities change over time, and your SEO strategy has to adapt. When you compare this 2015 version to previously leaked Google quality rater’s guidelines (as Jennifer Slegg does here and here), the differences point out how Google’s focus is shifting. The two biggest changes are:
- Mobile everything: Not only is there a whole new section devoted to mobile quality, but also most of the examples now show screenshots taken on mobile devices.
- Needs Met focus: A new rating scale judges how fully a web page result meets a mobile searcher’s need. Any site that is NOT mobile-friendly automatically fails this criterion. The entirely new section for judging Needs Met proves that Google is all about satisfying the searcher’s needs.
Here’s our checklist for making sure your SEO campaign aligns with Google’s priorities.
Mobile, Front and Center
Is your site really mobile-friendly?
Earning a passing grade on Google’s Mobile-Friendly Test tool is the bare minimum required for all web pages and apps now. Beyond this, you must make sure that tasks are easy to accomplish with a mobile device. From the guidelines, here’s a checklist you can use to evaluate how your site performs with a smartphone:
- How easy/hard is it to fill out forms or enter data?
- How does the site or app behave on a small screen? Are all features usable?
- Is the content legible without requiring left-to-right scrolling to read text?
- Do images fit on a small screen?
- How easily can someone navigate? Are menus, buttons and links large enough?
- What happens on your site when Internet connectivity is inconsistent or slow?
Needs Met or Not
How well does your site anticipate and fulfill a mobile user’s needs?
Another entirely new section added to Google’s quality rating guidelines is called “Needs Met Rating Guideline.” Here’s the description, which is clearly targeting MOBILE users’ needs (from Section 13.0):
Needs Met rating tasks ask you to focus on mobile user needs and think about how helpful and satisfying the result is for the mobile users.
To get a high quality rating in the Needs Met category, a search result and its landing page should:
- Require minimal effort for users to immediately get or use what they’re looking for.
- Satisfy all or almost all users looking for the same thing (so that they wouldn’t need to see additional results).
- Provide trustworthy, authoritative, and/or complete information that is helpful.
A mobile user’s intent differs from that of a desktop or even tablet user. (Tip: Aaron Levy’s SMX presentation covers mobile audiences in depth.) Evidence of this is found in the new mobile section of Google’s Search Quality Rating Guidelines, where page after page of examples show what mobile users probably want when they search for various spoken or typed queries. At one point, raters are instructed to “think about mobile users when deciding if queries are [a particular type]. Use your judgment here.”
The takeaway for mobile SEO marketers as well as for app/website owners is this: Think about what mobile users may be trying to do, and make sure that your site fulfills these things as directly as possible. Google is all about satisfying mobile users’ needs; you should be, too.
Answering this question takes some serious thought, but ultimately pays off in spades.
Does the webpage have a clear purpose, and how well is it achieved?
One of the first tasks a rater must do is figure out what a webpage is for and then decide how well the page achieves that purpose. For example, the purpose of a news site homepage is to display news; the purpose of a shopping page is to sell or give information about a product; etc. Google has very different standards for different types of pages, so understanding a page’s purpose lays the foundation for assessing its quality.
How helpful is the page’s purpose?
Google wants each page to be geared to helping users. Helpfulness factors heavily into quality ratings. On the low end of the quality scale would be pages that harm or deceive users (even though they may be fulfilling their designed purpose).
To be deemed high quality, a page must have a helpful purpose, such as:
- To share information about a topic
- To share personal or social information
- To share pictures, videos, or other forms of media
- To entertain
- To express an opinion or point of view
- To sell products or services
- To allow users to share files or download software
- … many others.
Is the purpose of the website as a whole clear, on and off site?
Make sure that your website’s overall purpose is explained clearly, ideally on the About page. The rating guidelines include examples of pages with “non-obvious purposes” — pages that seemed pointless or inaccurate on their own, until the rater referred to the About or FAQ page and discovered they were actually beneficial (see Section 2.2).
In addition, Google looks at independent sources to see whether the site’s reputation matches what it claims about itself. If there’s conflict, Google will tend to believe what the outside sources have to say. For small businesses or organizations, a lack of reviews or reputation information does not mean the site is low quality (see Section 2.7).
Meaty Main Content and Helpful Secondary Content
Does the page have quality main content?
A webpage’s main content (which excludes ads, sidebars, and other supplementary parts that do not directly fulfill the page’s purpose) can earn a high quality rating if ALL of these are true:
- There is a satisfying amount of high quality main content on the page.
- The page and site have a high level of E-A-T (expertness, authoritativeness and trustworthiness).
- The site has a good reputation for the page’s topic.
There are no hard and fast rules, and no minimum number of words per page. The guidelines encourage raters to decide whether the main content fulfills the purpose of the page satisfactorily.
Is there any supplementary content on the page that is helpful to users?
Google recognizes that supplementary content “can be a large part of what makes a High quality page very satisfying for its purpose.” Consider what you can include to offer related information, ways to find other cool stuff, or specialized content that could be helpful to people visiting that page.
YMYL Pages Have Higher Standards
How high quality are your site’s YMYL pages?
Pages that can impact a person’s “future happiness, health, or wealth” are known as Your Money or Your Life (YMYL) pages. Google first introduced this concept in the 2014 Search Quality Rating Guidelines, which held these types of pages to a much higher standard across all quality criteria. Examples include pages for shopping transactions, financial information, medical advice, legal information, and many more.
Google specifies “needs met” ratings that judge how well a webpage fulfills a searcher’s needs. If you have YMYL pages, needs met is particularly important.
Maintaining Your Site
Does your site look alive and well-maintained?
Raters are instructed to “poke around” to see whether a site is being maintained. Here are a few signs of life Google expects of a well-maintained, quality website:
- Links should work.
- Images should load.
- Pages should continue to function well for users as web browsers change.
How fresh is your content?
Google’s algorithm is known to look at “freshness” as a ranking factor for many types of queries. When Googlebot gets to your site, does it find any recently added or updated content?
For blog posts and other content that is dated, don’t try to game the system by setting up a program to automatically change dates to make things look recent; Google’s on to that scheme. Raters are even instructed to manually check the Wayback Machine to investigate suspicious dates to see whether content is copied or original (see Section 7.4.7). By the way, Google’s algorithm doesn’t need the Wayback Machine to recognize original content, so don’t even try to cheat.
A healthy website frequently adds new content and/or updates old content to keep things fresh and useful for site visitors.
How expert is your content?
We know from the 2014 guidelines that Google quality raters look for signs of E-A-T, which stands for expertness, authoritativeness and trustworthiness. The newest guidelines reinforce this concept, but define “expertise” differently depending on the topic of the page (according to Section 4.3):
- There are “expert” websites of all types, even gossip sites, forums, etc.
- Topics such as medical, legal, financial or tax advice, home remodeling, or parenting “should come from expert sources.”
- Topics on hobbies, such as photography or learning to play an instrument, “also require expertise.”
- Ordinary people may have “everyday expertise” on topics where they have life experience, such as people who write extremely detailed reviews, tips posted on forums, personal experiences, etc.
Make sure your expert content is “maintained and updated” to increase your site’s E-A-T rating.
If you have ads or monetized links on your site, are they appropriate for the page’s purpose?
The guidelines state that “the presence or absence of Ads is not by itself a reason for a High or Low quality rating” because Google realizes that many websites and apps owe their existence to ad income. However, Google “will consider a website responsible for the overall quality of the Ads displayed” (see Section 2.4.3). So keep an eye on the amount and use of affiliate, display, or other types of advertising. Make sure that ads don’t overwhelm the more useful main content (and supplementary content, if any) that each page contains.
Wrapping Up Your Quality Review
The old saying goes that there’s always room for improvement. This post is by no means a complete SEO checklist. We hope that as you apply these points from the 2015 search quality ratings guidelines that are based on Google’s priorities, you’ll begin to view your online properties with a new SEO point of view — and make your sites and apps more useful.
If you’re eyeing the best way to improve your website quality and would like to have a free consultation, fill out a quote request and we’ll give you a call.