What Should You Expect from a Search Engine?

Maybe the question is, why should you expect anything?

Bing-Duane-ForresterThe truth is, inside an engine is a busy place. I spent nearly six years representing a search engine to the SEO industry and I can tell you that, contrary to many search engine optimization conspiracy theories, neither Bing nor Google have designs on harming businesses. Both do have profit motives, however, just like you do.

Updates happen because … pause for drum roll … something undesirable is happening!

What Motivates Search Engines?

Research suggests that WordPress powers about 25 percent of the World Wide Web’s sites. More websites use WordPress, but of those using it, they are not “higher traffic” sites. OK, that makes sense — WordPress powers blogs mostly, we can loosely infer.

Let’s say, for example, someone made a WordPress plugin that managed ads across one quarter of the Internet. And that WordPress plugin placed the ads above the content on a page. Obviously, on an individual level, this is a recipe for increased revenue. But taken across the entirety of the user base, if even a fraction of WordPress sites adopt the plugin, it results in a disturbing trend.

If the engine is judged by users on the quality of the experience the user sees after clicking through on a result, then this places the engine in a difficult position. Do you police for quality or do you “throw it over the fence” and declare it not your problem?

But see, it is the engine’s problem.

Google faces ever-growing competition. Competition that’s gaining, albeit slowly, and in some ways seeking to change the game. Bing faces a similar problem on the other side of the coin — satisfying searchers enough to convert them.

In either case, the common ground is making searchers happy.

Reread that, as it’s a critical and often-forgotten point. The engines do not exist to satisfy a business. They exist to satisfy searchers. More subtly, they exist to develop revenue, to source data, and to feed that data into inter-related systems across the companies operating them.

When Do Search Engines Make Updates?

So when someone builds a popular plugin for WordPress, again, and it shows a trend that goes against the goal of happy searchers, the engines take an action. In this example, Google gave us the Page Layout Algorithm back in 2012. On Bing’s side of the graph, while the updates aren’t named or publicized, they happen. Indeed, they happen frequently. So frequently, in fact, that it simply doesn’t make sense to announce everything.

It’s important to note here that, in many cases, updates are not “to fix something that’s broken.”

Updates are a move from one state to a more advanced state.

all-about-usefulness-today-and-moving-forward-duane-forrester-400Maybe that’s a matter of perspective, but if you’re still asking me today about links, I’d say your perspective is wrong. Things change over time, and with thousands of smart people seeking to improve a product daily, things tend to keep moving. At the scale those projects work at, it can take time to manifest, but I’ve seen first-hand the changes at Bing.

If you think of Google as being a highway, every now and then you encounter a speed bump. It can be jarring. Everything is smooth, then suddenly WHAM! you hit an update. Bing, on the other hand, takes a different approach. The road surface at Bing is slightly less smooth overall, but once you’re up to speed, you don’t notice the slight surface irregularities. It feels smoother. Doesn’t mean things aren’t going on, just that your perception is it’s smoother.

This can help explain why ranking is harder in Bing at times, and more stable in the long run. And on that note, let the corrective comments, refuting examples and hate email start! I’m certain there are more than a few examples to rebut what I just suggested. But again, scale. When looked at not just across the sample set that even the biggest agency works with, the actual scale the engine operates at is enormously larger. In that context — the context of the engines — the statement makes sense.

How Do Engines Feel about SEO?

Does this all mean that the engines dislike SEOs, their tactics and businesses doing this work?

Nope. They readily recognize the improvements made by the work. The flip side is that the engines have to protect against gaming. This is why engines always take an arms-length approach.

The job of representing a search engine to the industry is not an easy one. I know. I was that face for Bing for about six years. Matt, Gary, John, Maile, now Christi Olson at Bing have all had to balance carefully. As individuals, all are approachable, helpful and knowledgeable. But that doesn’t mean any of us can give away secrets, or share things the engine wants kept secret. This is obvious, yet countless times at conferences the impossible is asked for, then derision heaped on socially when answers are not forthcoming.

Gary Illyes and Duane Forrester
As a representative of Bing to the SEO industry, Duane Forrester (right) answered questions posed by SEOs on stage with Google Webmaster Trends Analyst Gary Illyes (left) at SMX West 2015.

The reality in some cases is that due to the structure of the product, it’s completely conceivable the person you’re speaking to may not have access to the information, or that something changed since the last time they connected internally on the topic. Bing has a few thousand employees working on search. Google has, what, maybe 20,000 applied to search in some degree? I grew up in a city of 30,000 people, so asking me to know all the information and be current is like asking me to know the details on two-thirds of a city. Unrealistic. Even at Bing’s smaller scale, it’s an unrealistic expectation.

The bottom line here is there will always exist a push/pull reality. Engines want the improvements that optimization brings to a product, but they cannot be trusting to the point where results suffer. And in a space where searchers are growing ever savvier, ever clearer about expected results, it might seem like changes being made are punitive to businesses. But the reality is they are designed to appeal to searchers.

How Can Businesses Appeal to Search Engines?

Now, what might happen if your efforts were all geared towards improving the users’ experience, meeting their expectations and providing the highest quality result for their need? Do you think that might be something the engine would want to showcase?

So what should you expect from a search engine?

Exactly what they’ve given us for years. A lack of in-depth answers in some instances, rapid change and advancement, a searcher-first mentality, and a drive for revenue. We all know these are the ground rules. So why is it so hard for businesses to adapt?

There’s a better approach for businesses to take when it comes to SEO and rankings.

Don’t cater to search engines. Cater to users. Don’t settle for search engine optimization. Think bigger — user experience optimization. Internet marketing is SEO, PPC, content development, social media marketing, website design, and conversion optimization. Businesses are born to serve customers. Get back to your roots with your Internet marketing.

Duane Forrester formerly held the position of VP, organic search operations at Bruce Clay, Inc. Before that, he was a senior product manager responsible for Bing’s Webmaster Program. He's also the founding co-chair of SEMPO's In-House SEM Committee, was formerly on the Board of Directors for SEMPO and is the author of two books: "How To Make Money With Your Blog" and "Turn Clicks Into Customers." Connect with Duane on social with the links on his author page.

See Duane's author page for links to connect on social media.

Comments (19)
Filed under: SEO — Tags: ,
Still on the hunt for actionable tips and insights? Each of these recent SEO posts is better than the last!
Bruce Clay on May 8, 2024
The Always Up-to-Date SEO Checklist
Bruce Clay on April 4, 2024
What Makes an SEO Program Successful?
Bruce Clay on April 3, 2024
How Can I Improve My Website Rankings Through SEO?

19 Replies to “What Should You Expect from a Search Engine?”

Search engines are constantly improving their search queries for a better user experience! It’s imperative that SEO’s understand this because we need to help build a better web for tomorrow! :)

Every time, Google changes its algorithms – it causes a frenzy among those internet marketers relying heavily on the search engines. The latest change made many marketers stock up on Maalox and Pepto Bismol as it made them literally sick to their stomachs.

Great post. Thank you for explaining this point by point. I often used to think about the activities and rules of a search engine. Yes it is not a easy part to run a search engine in this market of competition but they are doing it well.

well said about search engine and its functionalities. and also we need more information regarding search engine updates and its uses in android platform. keep updating.

i am new to the word of search engine. here explains the motivation of search engines and with that details, Updation and details of search engines and how the SEO is used in search engines. Thanks for all these explanation.

We always look forward to Google algorithm updates. As creators and shepherds of business-to-business websites where the heart and soul is content that is optimized with on-site SEO (80% of which is just keyword-smart copywriting), we almost always see increases in organic traffic. This is because the “SEO companies” are focused on link building, which may not be black hat but is usually questionable at least to some degree.

I also agree that Google and Bing just want to do one thing: provide the highest quality search results for whatever you are searching for and to prevent people who game the system from appearing ahead of quality of content (gaming the system really boils down to doing the minimum amount of effort to get the maximum exposure – i.e. jump to the first result without quality content and onsite SEO best practices). The benefit to the search engines is that you come back and search more (and see all the AdWords ads).

One example of this, is that I like how they give extra keyword weighting in headings because people often skim headings to see what portion of a page they want to read. Therefore, headings are highly relevant and key to the success of a page and are weighted as such by the search engines.

I use mixture all at once to avoid tracking through my search history. How is this possible you all might ask? Easy! I use add-ons/extensions to get the job done!

But personally I use Duckduckgo, Startpage and Ecosia. These three combined give me the best worlds of Google and Bing with enhanced and customizable features for videos, images, instant and related searches, appearance, security/privacy (no tracking, no ads) and much more under the hood.

I employ Google’s search for the most part and used to value their search results: they made sense and I could find stuff and be on my merry little way. Now, I get more crap in search results that are not even related to my searches than I do for legitimate results.

A recent example was in a very specific search for a song my band plays. Despite that fact that the particular song (a top chart song from the 70s) is published all over the web, Google kept showing me YouTube videos of people covering the song — not instruction on how to play the song. They showed me membership sites that only offered to sell me subscriptions to access all their songs.

Google also could not seem to understand that I wanted sheet music, tab, a guitar tutorial — it just kept shoving other stuff my way and never did produce what I was looking for.

This is also the case in many of my local searches now. I Google something in Los Angeles and get results from Florida, Missouri — all over.

I understand (and loved) all your thoughts and agree that search engines strive to please consumers first — they should. But Google has made so many changes that it now seems to have made search results so variable and with all the wrong data. For example, I don’t want Google to factor in my previous searches, what I bough in Amazon or liked on Facebook. I want my searches to be clean and fresh from assumptions because multiple family members use my PC for their own stuff.

That said, playing by the rules used to be enough. Serving your consumers used to be enough but now, I feel like we are reverting a bit — it is harder to show in searches even when playing by the rules and making your site all about your people (and not search engines.) But what does show in search is often mind boggling in how wrong they got it.

I think you should also write an article on how users can get a better experience from Google. It’s only fair!

I completely agree with all the things you explain about SEO.
Thanks for your guidance, much helpful.

Awesome post about search engine logic.

There has been paradigm shift on how consumer is searching for products and services online. These behavioral science techniques are useful when it comes to google ranking algorithms.
Thanks for sharing. Great post.

@Thanks Duane Forrester for adding some extra very useful information in this awesome post.

Interesting read (comments too) :-)

@David – while the stats may be true, the fact remains that the engines need to focus on the entirety of the web when reviewing for trends. The stats do point out one thing…if you want to watch for negative trends as they emerge, WP could be a reasonable starting point. I know hackers like WP, which means the engines need to be careful, too.

@Joe – can’t say exactly why they haven’t adopted the Voice XML, Joe, but it might have to do with a few things. For example, if they’re already working on their own solutions, focusing on a different one is a lot of lost investment. Could be the protocol has limits the engine doesn’t like, could be they WANT to go proprietary. Fact is, we’ll likely never clearly know. might come down to simple math – while usage is growing, its not close enough to a tipping point to make the decision for them clear enough.

As for stats and data being shared? No business likes to share it’s data – it *can* be sued by the competition to learn a great many insights. Double edged sword there.

But the bottom line is this – I know a lot of folks at Bing and have met a lot at Google over the years. Not a single person seemed to be the sort to willfully harm another person or business. While changes may manifest that benefit some and not others, change happens.

If they don’t want to hurt businesses then why have none of them adopted Voice XML for the growing voice search results? Scraping an answer knowing that giving it to the user is likely to keep them from going to the website is hurting businesses. https://en.wikipedia.org/wiki/VoiceXML

And why are they not providing stats on it in their own tool sets or in standard analytics?

It’s a common misconception that WordPress sites “are not “higher traffic” sites”.

27% of the top 10,000 sites on the web use WordPress (that’s higher than the 25% penetration rate of all websites). WordPress helps power major brands like CNN, Time, Tech Crunch, and many many more.

High traffic websites are actually more likely to run WordPress than low traffic websites.

I am expecting my business to run proper. :)

I do agree with you the thing you explained about wordpress.

Google updates should always come time to time to filter the websites which ranked high in google but not relevant to the query being searched.

These websites gets higher rank in google by just doing good on-Page and Off page tricks. But infact that is the Black Hat SEO.

Hello Duane,
I found this post through google just and have read it fully.
You have just explained the truth behind search engine algorithm. It seems you are the real SEO expert. I like the way you related wordpress with Google search engine.

Awesome post!

Great article, Duane! I Couldn’t agree with you more. Focusing on the user gives SEOs a common target with the search engines, and that naturally will lead to better results because we are going after the same audience.


Your email address will not be published. Required fields are marked *

Serving North America based in the Los Angeles Metropolitan Area
Bruce Clay, Inc. | PO Box 1338 | Moorpark CA, 93020
Voice: 1-805-517-1900 | Toll Free: 1-866-517-1900 | Fax: 1-805-517-1919