Yahoo

June 11, 2009

Don't Let Shortcuts Create Bad Habits

Photo by Pink Sherbet Photography via Creative Commons

Google's getting pats on the back for a new tool that lets webmasters tell the search engine when a site's been moved to a new domain. According to Google's Webmasters/Site owners Help documentation, if you use the change of address tool, the Google index will be updated to reflect your new domain for 180 days. Before those 180 days are up, Googlebot will have come by to crawl and index pages at the new domain and -- voila! -- your new site will be forever present in the only search engine that matters. [Yay! --Susan]

It's great to hear that at least one task is being simplified for overloaded SEOs, but Google's convenient shortcut may have some unintended consequences. First of all, Google's not the only search engine. Submitting your new domain through Google's handy new tool is not going to affect the index of engines like Yahoo, Bing and Ask. (Rocket scientist here, I know.)

And second, while Google may be the big guy on campus today, that may not be the case tomorrow. Taking shortcuts can have a nasty habit of coming back to bite you. Just because Google's created an easy domain-change-submission process, that shouldn't be an excuse to ignore your responsibilities in optimizing a site for all the engines. After all, we're talking about search engine optimization and not Google optimization. Although, GO would be a pretty catchy title...

Anyway, if you catch my drift and you want to make sure your new site's sitting pretty with more than just Big G, check out this simple instructional on what to do after you've moved your site.

What to Do After Moving a Site

We recommend that all Web sites include an XML Sitemap. XML Sitemaps are used by search engines to find all the pages on a site. More information on XML Sitemaps guidelines and what XML Sitemaps are used for can be found in the SEO Newsletter archives. XML Sitemaps can be generated at http://www.xml-sitemaps.com/.

With your new site chilling until the search engines stop by, you'll want to submit the new domain's XML Sitemap to each engine according to its specific procedures. Now that I've explained what to do, here's the how.

Search Engine	What to Do	Submission Instructions
Ask	Ask does not have an XML Sitemap submission protocol. Instead, add the sitemap auto-discovery directive to the old site's robots.txt.	Ask Help Central
AOL	AOL uses Google's index, so follow Google's XML Sitemap submission protocol.	Refer to the Search Engine Relationship Chart
Bing	From Bing Webmaster Tools, select add new site, submit details of the site, including the address of the XML Sitemap, and verify the site with one of two available methods.	Bing Webmaster Tools
Google	In Google Webmaster Tools, add the new site by entering the URL and verifying ownership. Submit the new site's XML Sitemap. You can also use Google's new change address tool.	Google Webmaster Tools
Yahoo	Add your new site and authenticate your ownership. Upload the new XML Sitemap.	Yahoo Site Explorer

Posted by Virginia Nussey on 06/11/09 at 5:35 PM | Comments (3)
See more entries in AOL, Ask, Google, Live Search, Search Engines, Yahoo

May 14, 2009

Jumping into the Real Estate Fray, Part 2

It's day three of real estate week here on the Bruce Clay blog. We don't typically focus on a single industry in our coverage of all-things-SEO, but the current controversy over whether Google should be allowed to "scrape" listing information from IDX Web pages is a golden opportunity. It's timely, too, because this week thousands of real estate industry leaders have converged on Washington for the annual Midyear Legislative Meetings. Before I take the real estate industry as a case study and prescribe some SEO/SEM medicine to cure their ills, there's an update to the story!

Breaking News from Washington, D.C.

The announcement broke just hours ago, and Twitter was buzzing about it. It looks like NAR has seen the light! They've decided to change their IDX policy, which is used as the model for MLSs nationwide. Here's the news in a nutshell:

This means that NAR no longer considers Google a scraper! Another Twitter reporter snapped a photo of the policy change as it was announced:

In case you can't see it clearly, the revised policy reads: Participants must protect IDX information from unauthorized use. This requirement does not prohibit indexing of IDX sites by search engines.

So NAR sees the difference between unauthorized site scrapers and search engines. Hallelujah! This is a happy day for real estate professionals, who know they have been losing the Web marketing battle for years.

What Real Estate Professionals Should Do Next

Don't spend a minute wondering whether you should back out of the IDX data share. It's time to put that old fearful thinking behind and go full-steam ahead into Web marketing. Redefine yourself as a service provider and let go of the role of information hoarder. That self-image should be left crumbling in your trunk with the last edition of the old MLS book.

Instead, instruct your webmaster to allow your IDX pages to be indexed as soon as you hear the all-clear from your local MLS. And consider creating static pages with listing information in addition to your framed IDX searches and displays. You want to give the search engine spiders lots of indexable content to chew on!

And read on for more advice as I use the real estate professional as a case study for applying Web marketing and SEO.

SEO to the (Real Estate Industry's) Rescue!

Perhaps NAR's change of heart was the result of an "ah ha!" moment. Maybe, just maybe, they realized that by allowing REALTORS®'s sites to be better indexed, the industry might be able to claim more search engine real estate (pun totally intended). One can always hope.

Most real estate professionals understand that they desperately need to improve their online presence. But how? Despite the fact that most for-sale property information online originates with their listings, third-party Web sites get the credit. Run a Google search for real estate for any local area and you'll find at most one or two broker sites ranking on the first page of the SERPs. Primarily you find third-party real estate search sites, like Trulia.com, Homes.com, Zillow.com, Movoto.com, Yahoo! Real Estate, and the industry flagship Realtor.com. Many of these sites do direct traffic back to the listing broker's Web site if people click to "view more details," but not all. And there's no reason why a successful local broker's Web site should not come to the top when it's their neighborhood's real estate that's wanted! After all, they ARE the experts.

#1: Befriend the search engines. If you don't already, understand that Google is not your competitor, but your ally. When people search for a property in your area, you want them to find your Web site, right? Do a good job of letting Google know what you're all about. Put plenty of original text content on your site using keywords like "real estate," "homes for sale," "My Town Name" and lots of local area references, and that can happen.

#2: Submit your listings to Google. Brokers should be proactive andsubmit your listings to Google directly. You should be the one to get the information out there first. Consider submitting to the other search engines, as well. Yahoo! charges a hefty fee for this ($49.95 per listing), so check to see if your brokerage firm lets you do this on their dime.

#3: Don't hide your listing information. Many brokers hide listing information behind a form like this one:

Look at all those fields! Now I don't know how many people searching for home information actually take the time to fill out a form like that. Maybe it's working for some brokers, and they're really getting prospects this way. But what I do know is that search engines can't get past these forms. They'll act like a roadblock to search engine spiders, and your listing information won't be indexed. They may also be diverting users away -- after all, why fill out a form when so much listing information is easily available through the third-party sites?

#4: Build links to your site. I checked a bunch of broker sites' link counts. (You can do this using our handy free SEMToolBar, or by doing link: searches in Yahoo!) Not surprisingly, they had almost no inbound links. Seriously, the most I found was "3" links, and those were internal! If real estate is essentially a local business that builds credibility by networking within their communities, why isn't that network being reflected on the Web? As any SEO can tell you, without links to help establish your authority on a subject, you don't have a very good chance of showing up on the front page of Google.

One more point about building links is that not only do search engines quantify the links to your site, they also follow them. So if you're sure that for usability reasons you want to keep that roadblocking contact information form I mentioned before in place on your site, you can at least LINK to static pages all about your featured properties and other information. Spiders can then index those pages by following the links.

#5: Make your Web site unique. Many real estate professionals, knowing that they "had to have a Web site," took the easy route and set up a prefab template supplied by their brokerage firm or some third-party company. It's next to impossible to get indexed by the search engines if all your site offers is duplicate content, because it's the search engine's job to filter out duplicates and present searchers with the original, best, most authoritative version. And guess what -- that isn't going to be the cookie-cutter site. I loved the comment left by Foot In Mouth on our blog yesterday about this:

"I think real estate agents who just get a template based site and do nothing with it have a very expensive business card."

If you want your site to be found in the SERPs, you're going to need to make it different. Write about your local schools, neighborhoods, hang-out places, and your favorite frozen yogurt shop. Talk about new housing developments underway, changes in the rental market, and so forth. Give your site lots of locally specific information for the search engines to index. Then the next time Mr. and Mrs. Homebuyer go to Google and search for homes for sale in your area, your Web site might just be the first one that pops up.

Posted by Paula Allen on 05/14/09 at 2:15 PM | Comments (9)
See more entries in Copywriting / Content, Google, SEO Tips & Tricks, Search Integration, Yahoo

May 13, 2009

Jumping into the Real Estate Fray, Part 1

Photo by BlackHawkTraffic via Creative Commons

I'm feeling game today. Not having a snake pit to jump into or a wild animal to tame, I thought I'd venture onto the blog and dive right into the center of one of the messiest, most controversial issues confronting the real estate industry today. Why? Because it relates to SEO. And because it sounds like fun.

Real Estate Lessons to Learn

Virginia broached the dilemma facing real estate professionals in their search marketing presence yesterday. It deserves a closer look, because there are facets of this discussion that can be applied to every industry. Issues like content ownership rights and ranking for your own content may be red hot in real estate, but they have global application. So let's take a closer look at the problem, and tomorrow I'll outline a Web marketing solution.

Waving the Wrong Flag

We should be careful choosing a side to cheer for in the case of the multiple listing service that called Google a scraper and told some of its members to stop allowing certain MLS listings to be indexed. The broker whose Web site received the cease and desist order, Paula Henry, wants to make the issue about embracing technology and giving the public free access to information. She's raising the free-access flag that all of us love to rally behind. Her case paints NAR as old-school and against progress, and describes the local MLS as "an 800 pound gorilla" out to stomp on her rights. It's easy to give this a surface read and jump on her bandwagon. After all, we in the SEO industry are on the cutting edge of progress. We support technology. We're in favor of public access to information, and the more that can be indexed, the better!

But that's not what it's really about. This case is about Web content ownership, duplicate content, and who gets to rank for what in the search engines.

MLSs Are Not the Bad Guys

Multiple listing services do not stop Google from indexing property listings. They can control which information fields should be public-viewable (the banned list is usually short, things like agent-only remarks and showing instructions), but they actually encourage the online distribution of listings. Search engines cannot spider the MLS system directly because it's behind a login. However, selling brokers can and do advertise their listings on the Internet. Besides putting the information on their own Web sites, brokers can send it directly to public Internet sites such as Realtor.com, Google, Yahoo!, etc. Homeowners nowadays expect this extra online marketing, and many MLSs have even required their software vendors to provide easy ways to send new listings to these third-party sites automatically. I know, because I helped fulfill those requests while working for a leading MLS system vendor before coming to Bruce Clay.

Is Content Really Yours?

Paula Henry's indexed pages in question, however, were IDX listings that didn't belong to her. IDX is a different sort of thing. It enables showing other brokers' listings on your Web site.

A listing is like original content, researched and entered by the selling broker. Imagine you create a Web site with beautiful, original text. When someone comes and copies it, which always happens, you're rightfully miffed. If that person's Web site starts outranking yours for your content, though, you're justifiably angry. Won't you complain to Google and everyone else you know to try to get the index corrected? Or will you just roll over and surrender to the inevitableness of duplicate content?

When Broker Joe sends his own listings to Google, the search engine links back to Joe's Web site. That's perfect, no problem. However, if competing sites show Joe's listing information after receiving it through IDX and get indexed for it, it's Joe's loss, as least as far as his search engine marketing efforts go.

In the Web marketing world we know that it is often futile to fight for your rights when faced with content ownership issues. But that doesn't mean you shouldn't fight. Similarly, it is probably futile for real estate brokers to continue to demand ownership rights to listings and try to block other sites from being indexed for them.

Photo by Aaron G Stock via Creative Commons

I can't help wanting to root for their right to try. However, in the course of writing this article I've come to realize that blocking access just won't work.

So What's the Solution?

Real estate agents need to stop worrying about what to block, and focus on what to allow, instead. In tomorrow's post, I'll recommend some search engine optimization principles that the real estate industry needs to know and apply. Bickering in committee meetings over what Internet content to block isn't going to solve the real estate industry's dilemma. They need strong Web marketing and SEO medicine now in order to survive.

Posted by Paula Allen on 05/13/09 at 4:04 PM | Comments (7)
See more entries in Copywriting / Content, Google, SEO Tips & Tricks, Search Integration, Yahoo

May 12, 2009

Google Threatens Real Estate Professionals. Or Does It?

There's a debate raging in the real estate world. It isn't pretty and it isn't simple. The National Association of REALTORS® (NAR) has supported a regional association's decision that Google is a scraper site. The Indianapolis Metropolitan Board of REALTORS® (MIBOR) has issued cease and desist orders to a number of its members, telling them to implement robots.txt directives that prevent Google from indexing certain property listings. NAR has tentatively agreed with MIBOR's claim that the Google index is nothing more than a receptacle of stolen content.

Photo by Phil Scoville via Creative Commons

This week is the REALTORS® Midyear Legislative Meetings & Trade Expo. The annual event brings together NAR leadership, and it is expected that real estate professionals affected by the decision will ask the association to reconsider. The impending judgment is being closely watched by real estate professionals the nation over as the final outcome could either set a long-standing course for progression or futile resistance.

I consulted BCI tech writer Paula Allen, who worked for a real estate industry vendor for 12 years, for her take on why the NAR and other professional real estate associations are getting their pantaloons in a bunch. Aren't they aware that getting listings to show up as Google results is as good as free marketing, I asked? Is this just another case of overzealous, old-school types clinging to the past? Where real estate professionals were once the sole owners of listing information, was the solution to educate them on the power of search technology?

As Paula sees it, there's certainly a measure of that mentality getting in the way here. But there could be an even bigger problem that needs to be addressed before Google can be seen as a partner and not the enemy. To explain this requires a little background.

A multiple listing service, or MLS, is a handy cooperative that allows real estate agents to view and share listings with each other. Starting from the old days of the printed MLS book and continuing with today's Internet-based systems, access to the MLS data was long restricted to subscribers and is still governed by a strict set of rules about who can show which property listing information to whom. When competitive third-party real estate sites like Yahoo! Real Estate and Zillow started popping up, the MLS established the Internet Data Exchange (IDX), an authorized way to share MLS listings online. Those brokers who opt in to their MLS's IDX have their listings show up on other participating brokers' sites. In exchange, their site can advertise the listings of other member brokers as well. In this way, real estate professionals hoped to make listings available to people searching online while still maintaining some control over the information. But, apparently, it wasn't enough control.

As a member of the IDX, the displayed contact information for listings on your site is your own. So if broker Sally and broker John both participate in IDX, John's listing on Sally's IDX site will show up with Sally's contact info. That way, Sally will get the call, and potentially be able to represent an interested buyer. This is fair enough if someone searching Sally's site has come upon the listing.

However, things start to get fuzzy if that IDX listing is indexed in Google and someone finds Sally that way. According to the MLS, the listing doesn't belong to Sally; it belongs to John. John retains the control to put the listing where he wants. If he's smart, he'll submit his listings directly to Google and other search engine real estate services to get the traffic himself. And if he opts in to the IDX, then he has chosen to show his listing in that way. However, he hasn't allowed Sally to get all the credit for his listing in the SERPs.

Photo by FaceMePLS via Creative Commons

While on the surface it appears to be a simple "get over your power trip" diagnosis, there are a number of complicated implications in allowing Google to index IDX listings as they currently stand. I initially thought I'd berate NAR, demand that they embrace search engine technology as free marketing or be doomed to failure! But under the current system, indexed listings could be unfair to the rightful content owners.

The solution, in my assessment, will be two-fold. There will need to be a restructuring of the IDX system so that the contact info listed is that of the selling agent. Then, if IDX listings are indexed, credit will go where it is due. In the meantime, agents should embrace the power of search engine marketing and submit their property listings to sites like Google and Yahoo. This way they can proactively ensure they get all the inquiries. With an improved IDX system and a partnership mentality in place, search engine marketing education can take root. Down the road real estate professionals may just wonder what all the fuss was all about.

Update: For more on the real estate industry's search marketing challenges, please see our series Jumping Into the Real Estate Fray, parts 1 and 2.

Posted by Virginia Nussey on 05/12/09 at 5:32 PM | Comments (13)
See more entries in Google, Linking Strategy, Search Engine Optimization, Search Integration, Yahoo

April 27, 2009

Do You Yahoo?

It appears that Yahoo and Microsoft are still in the grips of a tangled tango, their long talks about a search advertising partnership remain hot and heavy. For more than a year now, the two companies have been in on-again/off-again discussions about a merger, partnership or some other strategic opportunity that would give the duo an edge against Goliath, aka Big G.

The two search platforms have historically struggled in the face of steadily shrinking market share. And in light of the current deliberations, it would seem that leadership on both sides of the table are looking for the windfalls that cooperation might garner. But rather than face the painful breakup of an all-or-nothing deal gone south, this time the couple is taking it slow with talks of partnership rather than marriage proposals.

In MarketingSherpa last week I read a primer on advertising with Yahoo! Sponsored Search. It just so happens that Yahoo's paid search program has a number of perks compared to Google's AdWords program -- not the least of which is a higher average return! A typical comparison for a single advertiser looks something like this.

Investment in Google: 80%
Investment in Yahoo: 15%
Investment in other engines: 5%
Average conversion rate with Yahoo: 3.42%
Average conversion rate with Google: 3.39%
ROAS with Yahoo: 2.85
ROAS with Google: 1.85
Percent of Google-driven online sales: 80%
Percent of Yahoo-driven online sales: 7%

Obviously, Google has a substantial advantage of market share and overall number of eyeballs, which leads many advertisers to ignore the solid ROI that Yahoo can provide. But as Kevin Lee explained during the Advanced B2B session at SES New York this year, "If you're not in Yahoo and Live, you're missing a chunk of your audience."

In the Bruce Clay offices, we see that our own clients are just as set in their ways. After talking to SEM analyst and BCI guest blogger James Kim, I learned that only 20 percent of our PPC clients are advertising in Yahoo. But as abysmal as that sounds, it's in line with the norm.

Why, I asked James, are so many clients missing the opportunity available through Yahoo and other engines? In his opinion it comes down to two things: market share and maneuverability.

"It is too early to say what effect a partnership would have on the advertisers. I think it will really depend on its interface and data reporting for PPC accounts. This is where Yahoo and MSN are seriously lacking right now. Google has made a lot of improvements to their PPC management, while Yahoo and MSFT are still trying to catch up on the basics. Once they collaborate, if they do it right with the right management tools, they may see an increase in advertisers."

Understandably, the day-in-day-out interaction an advertiser has with his PPC management tools can make the difference between a good day and a migraine. Google's intuitive interface is easier to navigate and includes more robust management tools.

As for the two areas Yahoo and Microsoft have to get right before they get anywhere, the companies will gain great strides in market share if they combine forces. A partnership might also result in the power players of each team coming together to design better management tools. With both market share and management tools wrapped up, we could be witnessing the debut of the next Silicon Valley power couple. They've already got the silly contracted name and everything.

Posted by Virginia Nussey on 04/27/09 at 5:15 PM | Comments (2)
See more entries in Google, Live Search, Pay Per Click / Online Ads, SEM Industry, Search Engines, Yahoo

April 14, 2009

Search Marketers and Web Developers Meet in Santa Monica

Update: Coverage of the topics and tactics shared during the event can be found on Search Engine Land.

Last night I attended L.A.'s very first Jane and Robot Web dev and search meetup. Nathan Buggia of Live Search and Vanessa Fox of NineByBlue.com have teamed up to bring the search and Web dev community a great resource in JaneAndRobot.com.

Nathan and Vanessa have presented a number of events like last night's all along the West Coast. Now they're gearing up for O'Reilly Found, a two-day conference dedicated to helping the Web development community understand their vital role in search acquisition.

I wanted to tell you about the good stuff that happened last night, but of course, pictures speak louder than words. So I'm thrilled to bring you photos by search documentarian Wm. Marc Salsberry.

To start us off, the host of the evening, Tony Adam, SEO manager at Yahoo, treated the 40 or so people in the crowd to yummy appetizers, cold brews and pizza!

Photo by Wm. Marc Salsberry - Entrepreneur/Photographer

Then Vanessa and Nathan powered up the projector and gave a presentation on how to diagnose a site's SEO issues. While half of the audience tweeted their up-to-the-minute thoughts via BlackBerrys and smart phones galore, this little blogger blogged.

Photo by Wm. Marc Salsberry - Entrepreneur/Photographer

Once the official Q&A had ended, it was time for the fun to begin. Vanessa, Nathan, Tony and the crowd headed to The Parlor for informal discussion and more good ol' fashioned networking.

Photo by Wm. Marc Salsberry - Entrepreneur/Photographer

Marc Salsberry is clearly crafty with the camera, but as an entrepreneur and photographer, he's also interested in marketing his photos and optimizing them for search. Marc has been posting his photos to Flickr and Facebook to allow for easy findability and open access to the community. But during Q&A time, he asked Nathan and Vanessa about the best way to get analytics and metrics for images.

It just so happens that Vanessa wrote an article on effectively using images on Jane and Robot.

Nathan also explained that the ideal situation for Marc would be to host the images himself. By hosting the images on his server he would be able to access the metrics and analytics he was interested in. If he linked from his site to the Flickr pages, Marc would potentially be able to secure two results in the SERP. And for branding and marketing purposes, Nathan suggested that it's important to have ownership of the URL where the image is located. All this occurs after deciding what primary goals and conversions he has for his images.

That was just a small piece of what went down at the Yahoo! Center last night. Bummed you couldn't make it to the Jane and Robot Santa Monica Meetup? It's not too late to sign up for the O'Reilly Found Conference going on June 9-11 in Burlingame, CA. Use the code FD09PC to receive 15 percent off the cost of registration! Thanks to everyone who made last night's great event happen and thanks to the talented Wm. Marc Salsberry for sharing your amazing images with the search community.

Posted by Virginia Nussey on 04/14/09 at 5:01 PM | Comments (2)
See more entries in SEM Industry, Search Engine Optimization, Web Development, Yahoo

February 12, 2009

Ask the Search Engines

Danny Sullivan, our fearless leader, moderates this panel; he's already in casual mode. Google's rep Matt Cutts, Google Inc., is here. Other search engine reps Keith Hogan, Ask.com, Sasi Parthasarathy, Live Search, Microsoft, and Priyank Garg, Yahoo!, are ready to sit quietly while Matt gets a billion questions. Too harsh? I'm just trying to be accurate.

Wow, big news.

Right before the session started, the big three announced a new tag rel=canonical. There's coverage popping up everywhere and I won't try to duplicate it so just get yourself over to the Google Webmaster Central blog and read about it. In talking to Matt, he emphasized that this is not a substitute for a proper redirect. If you can, do that first. Vanessa Fox covered it for Search Engine Land. Joost de Valk does a good job of explaining what it means over here Canonical URL Links, and he's already got plugins ready for this for Wordpress, Magento and Drupal already. Can you say on the ball?

Matt's going to do the official introduction now so if those links aren't enough, you can try to follow along here.

Duplicate content is the bane of a lot of people's existence. There are many many preferable ways to fix duplicate content: Fix your CMS, link consistently, make all non-canonical URLs redirect, etc. However, that doesn't mean that you're going to catch every single instance. Even Her Majesty the Queen has dupe content issues.

So they've come up with a very simple link element that you put in the head section that identifies that clean, pretty, preferred URL. It's .

They'd rather not have to resort to this. Do everything else first. This is a hint, not a mandate. They'll choose. And they reserve the right to treat spammy uses of it as spam.

This only works on the same domain. It does work across subdomains/host. You can use it for https versus http. They don't have to be 100 percent identical but the differences should only be slight. You can use relative or absolute URLs but they'd prefer you use absolute URLs. Google can follow a chain of canonicals but... you know, don't do that.

What if I point to a 404 or have an infinite loop or an uncrawled URL or a non-www/www conflict, etc.? Don't cross the streams. They're going to handle it as best they can.

Thanks go to Joachim Kupke for doing the heavy lifting at Google to implement this.
Okay, ready? Time for questions.

What are you doing about the problems related to deep crawling?

Matt: We try to crawl as best we can. We look at PageRank and backlinks and if you have a higher PageRank, we'll try to crawl deeper. If you're not getting crawled, you might need more links. Also, check for duplicate content and make sure you have a simple site structure.

Priyank: Having a clear site map helps Yahoo. The link canonical will work very similarly to a 301. Be careful; don't mark unique content with a link canonical.

Sasi: Those two pretty much summed it up. Ease of navigation and site architecture is important. At the end of the day, if it's good content, they want to crawl it.

Keith: We go a step further and look at the content. If it's good content, we'll crawl faster. If it's bad content, we're going to slow up and not crawl as much.

Should the new canonical tag be used on pages that we don't think there's a problem with?

Matt: Yeah, you can but be careful about it. Think about what the user is going to want. It is just a hint though so if you pick the one that we think is not as good, we reserve the right to pick the URL we think is best.

Priyank: Be careful. We all want to emphasize that. Use it with caution.

If someone under a penalty 301s to you they can pass the penalty. How do you protect from that?

Matt: We do look at it. We try to be fair.

Does Yahoo have a hard time with special characters like pipes and ampersands in Title tags?

Priyank: That's a very specific question but no, I don't think so.

How is the canonical tag different than IP-based cloaking?

Matt: One is a mini 301 and the other is not in any way.

Why doesn't the canonical tag work across domains?

Matt: You could use it across domains but it would be hard and it could be a little unsafe, so we decided not to implement it like that.

Will the new tag slow down crawling?

Matt: It won't slow it down, but we don't follow 301s immediately anyway so it'll just get stuck at the end of the queue.

Priyank: Same. We also have dynamic URL in Site Explorer so that you can tell us if you have a session ID or something that can be dropped while crawling and that will speed up the crawl.

Let's revisit nofollow. Do nofollow comments mean no impact or help to you?

Sasi: It's not helpful from a search engine perspective, but it may be from a user perspective. We just ignore them.

Matt: We drop it completely off the link graph, nothing flows, no PageRank.

My competitor is spammy. What is going to happen to them?

Matt: White text on white background is definitely spam. Catch me afterward, I'd love to hear examples. [There's much laughter and Danny uses this to segue into...]

Google Japan has been buying links. What the heck, Matt?

Matt: I just want to take a minute and apologize for it. They didn't think about how it was going to affect search engines but that doesn't excuse it so they were lowered from 9 to 5 to reflect the decreased trust in Google.co.jp. They have to file a reconsideration request, just like anyone else. There are a lot of people at Google who were angry about that but that was the right thing to do.

How does the message of what you can and can't do get out?

Matt: I think we took a very clear position that this is bad. I've been thinking a lot about this the last few days. Google can't just assume people know, they have to keep talking about it and they still have a lot of communication to do about it.

How do you determine what's natural and what's not?

Keith: We look at each individual page, at the content.

Sasi: We use neutralization as well as penalization.

Priyank: We run analysis on every page. Cloaking is remarkably ineffective.

Matt: All of the above, particularly what Sasi said. Google's more likely to penalize sellers than buyers because it's usually more clear cut, but they will penalize for it if they have a high degree of confidence.

The new spam technique of 2009 is going to be hacking sites. You need to keep your servers patched, watch your open redirects and monitor them. Watch your server load. Keep your software patched. If you don't have a recent version of WordPress, you're going to get hacked. Stay patched.

Priyank: I agree with Matt. The onus is on you.

Sasi: Security should be a primary focus for webmasters. It's your page, not anyone else's. You'll end up getting the penalty.

Matt: Danny had a really good point about "craphats" and how it's like racist jokes -- they're that out of fashion. Don't put up with it.

My competitor is coming up as a "did you mean" for my site! How do you stop that?

Matt: We do pushes every few months so sometimes that fixes it. OR you should do a blog post and call Google an idiot for getting it wrong. [Hee!!!] You can also report it via a feed.

All others: We want to know about it and want to fix it. Give us feedback.

Will you notify webmasters if you see them get hacked?

Yahoo does already and so does Google. Ask has a partnership with Norton for notification. If you get flagged in Yahoo and you're not malware, there's a link right there in the warning that you can respond. Google and Microsoft will tell you if you've been penalized as well.

Matt: I want to know from those in the audience. Would you want a notification in case of hacking or penalization? Do you want an email? (The audience says YES.) Should it be a check box? (Another YES!)

Priyank: How many people target content at a state, regional or city level? Language? Country? [I can't see the responses.]

Matt: Would people want a fetch as googlebot feature? (The audience says YES.)

Priyank: How many people would like to have their content crawled at a specific time of day? [No one seems to care. Except Hilton.com who doesn't want to be crawled during the day. Um... yeah.]

Posted by Susan Esparza on 02/12/09 at 3:03 PM | Comments (1)
See more entries in AOL, Ask, Google, Live Search, Liveblog, SMX West 2009, Search Engines, Yahoo

December 1, 2008

Holiday Shopping Made Easy Through Search

I hope everyone had a delicious and relaxing Thanksgiving weekend. I always love Thanksgiving because it hastens the final and most celebratory part of the year -- the holidays! That means sparkling lights lining the streets, sweet, bright melodies that make your soul smile, warm, spicy scents wafting, and for some, presents large and small meant to symbolize the love that has been collecting over the year.

There are always a few people on my list that completely stump me. What do you get the mom who wants nothing? The dad who has everything? The broke college student sister who needs everything? What about the teen whose taste requirements far exceed your own? Every year is a struggle. Luckily for folks like me, the search engines have rolled out some cool features to make getting that perfect gift a little easier.

Everyone's looking for a bargain this holiday season and Live Search is ready to give you one. Building on its cashback program, Live Search is now offering instant cashback when shopping on eBay. Eligible shoppers who make their eBay purchase with PayPal will have their cashback automatically deposited into their PayPal account.

If, like me, you get hung up on what to get your special someones, Google has created a site where all the most-searched-for gadgets, gizmos and gift ideas are waiting for you. Based on recent Google Product Search queries, video games and toys, cold weather clothing, specialty foods and high-tech devices have been rounded up to create the Google holiday shopping guide. Plus, through that site you can redeem special offers when you buy through Google Checkout.

Yahoo's got a similar offering, using search trends to find the must-have gifts of the season. As the close of the year rolls closer, Yahoo is presenting the top ten searches of the year along with a number of top ten searches in niches like news, politics, the economy, celebrities and more. For those looking for gift ideas, movies, tech products, games and shopping top ten lists provide a wealth of information on what your loved ones are drooling over.

The most cheery time of year is upon us. Be free with your affection and fuzzy, happy thoughts. (If you need a lesson, talk to the amazingly generous Susan Esparza or Lisa Barone.) Remember that size and price doesn't matter -- it's the thought that counts. And when you've run out of those, search trend data can keep them coming.

Posted by Virginia Nussey on 12/ 1/08 at 4:47 PM | Comments (1)
See more entries in Google, Live Search, Yahoo

November 18, 2008

After Yang Steps Down, What Are Yahoo's Next Steps?

Yesterday, Jerry Yang, CEO of Yahoo, announced that he would be stepping down from his position to return to the role of chief Yahoo. In a letter to employees, Yang said:

last june, i accepted the board's request that i assume the ceo role to restructure and reposition the company as a whole in order to more effectively meet the fast-changing needs of both users and partners. since taking on the ceo role, i have had an ongoing dialogue with the board about succession timing. thanks in large measure to your tireless efforts, we have created a more open, competitive yahoo! and we believe the time is now right to transition to a new ceo who can take the company to the next level.

Yeah, I'm pretty sure your stock holders agreed that the time for change had come. Actually, I know so. TechCrunch reported that Yahoo's stock (YHOO) went up 12 percent -- or $1.83 BILLION! -- the morning after the announcement. So yeah, I guess more than a few people thought it was about time.

There's been a good amount of speculation over who will be courted as Yang's successor, but the question remains. Is Yahoo looking for someone whose specialty lies in technology or in media? Or even entertainment -- although, the last time they tried that, things weren't any better for Yahoo than they are today.

Like most people, I'm inclined to think that Yang's decision was in the best interest of Yahoo -- a company that has struggled to find market share under the shadow of a behemoth.

When Yang first stepped up to the position of CEO, there already was muttering that the role might be a temporary one. In the postscript of this story from when Yang was first announced as Yahoo's CEO nearly 18 months ago, Danny Sullivan points out that Yang was looking at the position from a long-term perspective and that Yang's intense passion was (and is) one of the best assets he offers the company. Now with the advantage of 20/20 hindsight, it appears that that passion may have also been his undoing.

It's undeniable that passion among executives and low-level staffers alike is vital to the health of an organization. It's that passion that makes Yang the best possible chief Yahoo and brand evangelist around, and it's that passion that drove Yang to found the company to begin with. When it comes to business decisions, however, sometimes the best are made with a level-headed objectivity most often found far from emotional impulses.

Of course you don't want a limp fish running the show, but the successful CEO is able to tap into and out of his or her emotional well as the circumstance demands. (Please don't go locate the closest zombie to run your company. That's not what I'm suggesting.) Likewise, the best candidate will have a solid background in both the tech and media industries -- a position that will enable him or her to navigate the company back to the position of number one Google threat. It's a long road, but with the right captain, the journey will be worth it.

Good luck finding your new CEO, Yahoo. We're all rooting for ya.

Posted by Virginia Nussey on 11/18/08 at 5:15 PM | Comments (4)
See more entries in Yahoo

November 13, 2008

Getting Rid of Duplicate Content Issues Once and For All

No fancy intro here, just right to the content. The moderator for this panel is Rand Fishkin. Speakers are super funny Derrick Wheeler, Senior Search Engine Optimization Architect, Microsoft; Ben D'Angelo, Software Engineer, Google; and Priyank Garg, Director Product Management, Yahoo! Search. Rahul Lahiri, VP of Search Product Management, Ask, is a might show. Hmm.

Ben D'Angelo is up first. He's been with Google a little more than three years. I think that means he went to Google straight out of grade school.

What are duplicate content issues? There are actually multiple disjoint problems.

• Duplicate content within your site or sites:
  • Multiple URLs point to the same page or similar pages
  • Different countries (same language)
• Duplicate content across other sites:
  • Syndicated content
  • Scraped content

The guiding principle behind the search engines' indexing is ONE URL for one piece of content. Why? Because users don't like duplicates in results. It saves resources in Google's index, leaving more room for other pages from your site. And it saves resources on their server. [So Ben is telling us to keep duplicate content low to save Google money? Man, that stock price must really be suffering.]

Sources of duplicate content:

• Multiple URLs pointing to the same page
  • www vs. non-www
  • Session IDs, URL parameters
  • Printable versions of pages
  • CNAMEs
• Similar content on different pages
  • Manufacturer's databases
  • Different countries

How does Google handle this? They cluster like content and pick the best representative. There are variations on this depending on where it is in the pipeline. Different filters are used for different types of duplicate content. In general, it's just a filter and it's not going to destroy your site.

The problem comes in when Google doesn't choose the page you want or makes a mistake in clustering. You need to take back control.

Use 301 redirects for exact duplicates, like tracking URLs, and to solve www vs. non-www issue. You can also address exact duplicates in Google Webmaster Tools, but that only solves the problem for Google. He demos briefly.

For near duplicates, no index or block with robots.txt. Things like printer pages and site clones should have this.

Domains by country are a little different. Different languages are not duplicate content. Same language, different country? Don't worry about it -- the right one will usually be okay. You can geo-target in GWT or use different TLDs to help Google recognize where the content belongs. Best of all is creating unique content for that country.

Leave out URL parameters if you can. Put that data into a cookie instead.

In Webmaster Tools you can check for all sorts of other problems too, like duplicate Title and Meta data. Fix those things.

If another site has content that duplicates yours, there's less that you can do.

Duplicate content from syndication should include a link back to your site to make the canonical origin clear. Another option is to syndicate different content than what you publish on your site. If you're publishing content you have syndicated, manage your expectations.

Don't worry about scrapers or proxies too much. They generally don't affect your rankings. If you're concerned, file a DMCA request or a spam report with Google.

Duplicate content best practices:

• Avoid duplicate content in the first place.
• Generate unique, compelling content for users.
• Don't be overly concerned with duplicate content.
• Let us know about any issues at the Webmaster Help Forum.

You can always check out the Webmaster Central Blog and check out the Webmaster discussion group.

Priyank Garg is next up. He's got a sore throat so he'll be brief. His voice is all scratchy. Aw.

Much of this will be similar to Ben's presentation -- I'll pull out the Yahoo-specific stuff. Like Google, Yahoo filters at several places in the pipeline. Session IDs and other "content neutral" parameters can really hurt your crawl queue. They might never get to the rest of your content because they're crawling the same page over and over with a session ID. "Soft" 404 pages can also cause duplicate content problems. Repeated elements (perhaps with just a keyword replace) lead to problems.

Abusive dupes include scrapers/spammers, weaving and stitching, etc.

• Slurp supports wildcards in robots.txt.
• Yahoo Site Explorer allows you to delete URLs or an entire path from the index for authenticated sites.
• Use the robots-nocontent tag on non-relevant parts of a page.
  • Robots-nocontent can be used to mark out boilerplate content
  • Robots-nocontent can be used for syndicated content that may be useful to the user in context but not for search engines.

You can do dynamic URL rewriting in Site Explorer. Tell them which parameters are content neutral for your sites:

• Ability to indicate parameter to remove URLs from site
• More efficient crawl with less duplicates
• Better site coverage as fewer resources are wasted on duplicates
• Fewer risks of crawler traps
• Cleaner URL, easier for user to read and more likely to be clicked
• Better ranking due to reduced link juice fragmentation -- it's equivalent to 301ing all the duplicates back to one URL, saves time because they don't have to crawl it

Derrick Wheeler is up. Here's a bit of vintage Derrick for you all: "This crowd is a perfect Web site. You're all unique. I would crawl, index and rank all of you." Rand interjects "That's dirty." Derrick: "But I wouldn't click or take action." Hee.

Final points (he likes to get these done first):

• Consider search engine crawler detection
• Know your parameters
• Link to URLs with parameters always in the same order
• Dig deep into search results for your domain
• Exclude duplicates by robots.txt first, Meta Robot exclusion second, and nofollow link attribute last
• Don't assume engines can't follow JavaScript
• Get a regular crawl report of your Web site
  • Request a tab file that includes: referring URL, fetched URL, redirect path with type, landing URL with status code, Title, Meta Description, Meta Keywords
  • Open file using Excel 2007, sort by Title then landing URL
  • Review suspect URLs to look for dupes
• Focus on your strengths

Look for spider traps, adding a parameter and creating new pages every time you go back and forth several times.

Make sure that when you're creating sites for users, you still avoid spider traps. Just because you don't think the search engines will need to index it, doesn't mean that you don't have other pages that the search engines won't get to because they're busy with your trap.

Document why you're doing things. One site removed session IDs for search engines and got 10 million pages indexed. Down the line, someone forgot why it had been done, started giving session IDs to the engines again and their index pages plummeted again.

Look for things that might be causing problems, like dynamic breadcrumbs, based on how someone clicked through the site (Brookstone does this), related products, etc. They might be helpful for users but you're probably going to get into trouble. Make your internal linking consistent and useful. Some products might be able to live in multiple categories, but you need to make a decision.

Anytime you see related, sort or compare, think "possible duplicate content". When you see "select region" or "sign in", think duplicate content. Disallow those pages in your robots.txt. "Email an article", "send to a friend" -- think duplicate content.
Once you screw up the parameter order, it's hard to fix. Keep it consistent.

Use absolute links, not relative links, especially when switching between http:// and https://. Other people could link to you with https:// as well and you can't really do anything about that.

Priyank suggests going after the low-hanging fruit. Try the dynamic URLs first so that you can see the benefit right away.

Brent Payne asks: How do you credit a story properly when you're the Chicago Tribune? Can I get a link attribute or something? Just linking back doesn't work. Google tells me it's not a big deal but it is.

There's not so much that the reps can say to that. They're trying and he's already doing the right thing. Poor Brent.

Derrick doesn't think there is a solution right now. (He also reminded everyone that he's an in-house SEM, not a search engine representative.)

How detrimental are different link IDs?

Priyank: Every different URL linking to the same content is duplicate content. That's why you should use dynamic URL rewriting.

Ben: We try to handle that automatically. We might have to crawl the page once but we try to learn which parameters don't affect the page content.

[Most of these questions are site specific, so I'm skipping them.]

Posted by Susan Esparza on 11/13/08 at 12:06 PM | Comments (1)
See more entries in Google, Liveblog, Pub Con Las Vegas 2008, SEO Tips & Tricks, Yahoo

October 10, 2008

Ask the Search Engines

The last day of SMX East was hectic as I tried to touch base with new friends and contacts, attend all the sessions and make it to my flight in time. I managed to pull off most of those objectives, with the exception of posting a couple of live-coverage entries to the blog. So without further ado, here's what happened at the last session I attended. As my momma always says, "Better late than never!"

Moderator Danny Sullivan has got to feel good right now. Another awesome conference nearly complete! For the final leg of the marathon, let's go straight to the source and talk to the search engine representatives about all the things on our minds.

Danny says that he used to do a session called "I'm So Confused" because of all the conflicting information that is shared at conferences. But this panel will give us the official take from the search engines.

The reps are Nathan Buggia, Live Search Webmaster Central, Lead Program Manager, Microsoft; Aaron D'Souza, Software Engineer, Search Quality, Google Inc.; and Sean Suchter, VP of Engineering, Yahoo.

Sean says that sites should submit sitemaps, either .txt or .xml, and overall it helps with inclusion. He also heard a question about keyword order in Titles. He says that it is important to get right, not because of ranking but for the effect of the presentation in the SERP. Users will react well to seeing the keywords they are searching for in the Title so they should be further toward the beginning.

Aaron says that he's involved in trying to get rid of spam. He hears a lot about companies wanting to put up different versions of content for different countries. They wonder if it's going to be a duplicate content issue. He says that if the URL and the path to the content is reported to Google as specific for a certain location, Google won't see it as duplicate content.

Nathan says that he hears a lot about URLs. He says that session tracking parameters for a page will result in multiple versions of the same page in the index. Competing against own pages for space in the index can be harmful. He recommends submitting a sitemap with one URL for each page, and it should be the shortest form in the canonical version consistently. He also hears a lot about metrics and thinks that people are worrying about metrics that aren't the most important. He thinks it's all about conversions and trying to find the most valuable action. Finally, he doesn't believe that enough people are using the search engine provided tools available.

Now comes the Q&A part that you've all been waiting for. Be kind; Q&A can be hard to blog.

Are there best practice for running A/B tests so search engines don't think you're trying to cloak?

Aaron says that the way they look at it is that cloaking is only a problem if the intent is malicious. So for A/B testing, it is fine because the same type of content will be served. While they don't encourage cloaking, penalties only happen after a human review, so no penalty will be served if it's clearly just testing and not malicious.

Nathan says that A/B generally looks different than cloaking, and while they don't recommend cloaking, it really isn't a problem.

Sean says that the bad situation happens when there are large diversions, not the little ones that are common of testing.

Do you count affiliate links?

Sean says it depends where and in what context the links are coming up. If they are coming up in random, irrelevant places, that's not good. But if affiliates are making them of value to users, it's probably going to be a fine signal.

Nathan says that each link is evaluated independently and it's not necessarily considered if it's an affiliate or not.

We're currently redesigning our site and the only thing staying the same is the domain. The old site had ten pages and the new one will have 100,000. Are we going to have a problem?

Nathan says that the search engine always tries to find the most relevant page for a query, so if there's a page with similar content about a product as the manufacturer's page about the product which has been around longer, your site's page may not show up as it's considered a duplicate. One way to work around this is if you can add something beyond what's already out there, like pictures or reviews. As the question came from someone who has a weapons site, he suggests that she could maybe do videos of tazing pets... The whole audience laughs and groans and I'm pretty sure Nathan is turning pink. Maybe not the best example!

Aaron says that when you have a unique offering in a market, you will stand out by doing something different. He doesn't think that the reputation credited to the old site will be devalued on the new site, but he does warn to be aware of duplicate content from the old site.

When will Yahoo and Microsoft get country-specific targeting? And what's your advice if you want your site seen in another country?

Sean says that you should use a ccTLD because it's a huge signal. The other big signal is where the users and links are coming from.

Nathan says that you should make sure the international site is all located in the same sub-group or sub-directory because it's easier to identify. If a whole sub-directory looks like it's in German, it's a signal that it is targeted for Germany.

What percentage of false positives do you have in spam protection?

Aaron says that it's low but that it's an algorithm so there are sometimes mistakes. Spam algorithm changes are treated the same way as any other algorithm change. They test changes in a large sample and if they see a generally overwhelmingly positive result, they roll it out.

Sean sways that it's low, but if you think your site is treated incorrectly or if it has been cleaned up, submit a webmaster support form for consideration by the right people.

Danny says that Microsoft and Google will report to you if they think you're spam, except for cases that Google feels are so obvious the site is spammy that you should know it already. Yahoo is working on it.

Should people be bothering with nofollow or not to try to flow their PageRank around?

Sean says that in terms of designing for users, it's not helpful at all, so in the long term your energy could probably be better put into other areas. Aaron says that for the most part the issue comes up when there are way more links on a page than are useful to a user. In that case you have to think if the page itself is good for the user. He doesn't think it's going to cause an issue one way or another. Nathan asks who in the audience is doing sculpting (maybe five) and then he asks who has measured a positive change (maybe two). He says it was higher than he thought, but he still doubts the long term value.

Aaron says that sculpting seems like a lot of effort to put into the one signal of the link equity algorithm. He says he thinks it can be done if there's nothing left to do. Danny recommends testing it yourself to see if you see a difference.

It was suggested in a link building session that you could make donations to charities to get a link on their .org site.

Danny says that, to make it more uncomfortable, Matt Cutts has said that's fine. Sean says that if a charity is offering links for sale, he would think that they'd be getting links from bad guys as well as good guys, which will quickly get them flagged. Then the site will be in the universe of people who are bad and that link will be worthless.

Aaron says that if they were to see that 60 percent of the spam comes from charities, then they'll go after it. If it's rampant and makes up a large portion of spam then they'll see it as low-hanging fruit. Nathan says that if you're giving it to charity, then it's good anyway. But really, a charity that is aggressively selling links is probably going to see other attention as a result of their marketing techniques and see an increase in traffic.

Do you ever do direct intervention to penalize spam, as opposed to changes to the algorithm?

Aaron says absolutely. If it's hurting the results right now then they're going to do something manually. But they want to make the algorithm better, too, which they do by learning about the ways people are spamming.

Do reports that come in from a Google account have more weight?

He says that reports that come in from Webmaster Central are considered first over the external submissions because it's a cleaner data set.

Does the Yahoo algorithm in Japan work in a significantly different way than in the U.S.?

Sean says that there are slightly different signals but that it is the same back-end search engine and system, just tweaked for the market.

In natural search, do you offer some sort of endorsement or certification for SEOs?

Nathan says no. He says that he wouldn't want to endorse vendors because there's so much behind it. Sean says that it's the second time he's heard the question and says it's an interesting suggestion.

Is there a conflict behind your content networks showing up in your search engines?

Sean says that the reason Yahoo has SEOs is because they're trying to avoid a conflict of interest. There's search and there's content and it's not the same thing. So, for the content they have to compete for their user base and thus they need SEO. Aaron says that there's no Google policy to boost Google properties, but for certain properties like YouTube they have more information on them than they have on other sites, so they may show up more. Nathan says that Microsoft tries to keep a firewall between all of their businesses. Even advertisers that spend tons of money get no preferential treatment. AdCenter and the search engine are separate.

Are links still the primary signal for popularity and importance?

Aaron says inks are a good measure of reputation. Clicks are a noisy signal, and so the absence of a click for a result is thus way more useful because it signals that it's not the most relevant result. Sean isn't sure if links are the most important signal or not, but he will say that it's a larger signal than Title tags, for instance.

What's happening with personalized search?

(Okay, I actually didn't hear the question, but this is the answer.)

Aaron says there's a lot of data they have access to because of the way people use the search engine. But in personalized search, one policy is that whatever is done will be told to the user. The user can go in and control what is being used for personalization. They want to give you the ability to say "I don't want you to use this".

And that's a wrap for SMX East! Thanks to Cindy Krum, Eric Lander and Kate Morris, who took time out of their whirlwind schedules to come on SEM Synergy and, of course, thanks to all the great speakers who didn't hold anything back when it came to sharing with hungry audiences. All that's left to post from the conference is the highly-attended Give It Up: White Hat Edition panel, which will be hitting the blog November 7.

Posted by Virginia Nussey on 10/10/08 at 1:17 PM | Comments (4)
See more entries in Google, Live Search, Liveblog, SMX East 2008, Search Engine Optimization, Search Engines, Yahoo

October 7, 2008

Googleopoly

Finishing out the day on the Issues Track, let's take a look at Googleopoly! Our moderator is Jeffrey K. Rohrs, VP, Marketing, ExactTarget, and he'll be the one asking the questions. Our speakers are: James Grimmelmann, Associate Professor of Law, New York Law School; Shelly Palmer, Managing Director, Advanced Media Ventures Group LLC; Kevin Ryan, CEO and Founder, Motivity Marketing; and Jimmy Wales, Founder, Wikia Search.

What does it take to be considered a legal monopoly?

James says a monopoly is simply when you're the only one selling something. There's nothing illegal about having a monopoly. The thing is, you can't do unfair things to get one and once you have one, it's not legal to exploit it to discourage competition.

Are you hearing concerns from advertisers regarding the Google Yahoo deal?

Of course advertisers are concerned because they assume a partnership between Yahoo and Google is intentionally vague so as not to be understood. The world doesn't know what it doesn't know, and people generally don't know that having one source of information on the Internet is a bad thing.

Does Google's growth in the domestic market concern you?

Chris thinks we're going to see competitive responses. In China, Microsoft revealed a new suite of tools that they're going to have which show quite a bit of transparency. He doesn't think that Google's monopoly is bad, and he sees competition coming about.

Hitwise is reporting that Google has a dominance in the video space, which appears to be propped up by the search space. Is that concerning?

Shelly says there several things to think through. First, Google is an ecosystem and it's not likely to go anywhere. The chart shows that Google, if left unchecked, is pretty much unstoppable. But not only is Google a medium, it's a metric. That's a new place advertisers are finding themselves. Google is the metric of how to do other media that has become ingrained and will be hard to unseat. He's only concerned from the perspective of how advertisers will handle the circumstance after never having seen this kind of shift before.

Now Jeff pulls up the Wikipedia page for "Googleopoly" and it's a sad little page. (Go at it, marketers!)

Share your thoughts on Google growth and how it fits into your view as a competitor.

Jimmy says that search doesn't trend naturally toward monopoly. Like big brand spaces, there are reasons why some are going to be big and others are going to be small. He says if he can launch a search engine he's going to be super thrilled to get even 2 percent of the market. To launch in the advertising space is much harder.

Jeff shows a chart of one company's client's paid search spend for Google, Yahoo and Microsoft. Google is up at the 70 to 80 percent line, Yahoo is around 15 to 25 while Microsoft is hovering around 5 percent. So what does this mean for the monopoly issue?

James says that from a market share perspective, Google is crossing the point of being considered a monopoly. At around 75 percent, some could start seeing monopoly, but the real question is if they are doing anything to clobber competition.

Jeff asks the audience who is an advocate for the deal, no one raises their hand. Almost everyone raises their hand when he asks who is a skeptic.

Advocates

• 11 members of Congress (CA)
• Ayn Rand Center
• Google
• Overstock.com
• Publicis Group
• Randall Stross (Author, Planet Google
• Yahoo

Skeptics

• American Antitrust Institute
• ...

Ugh, the slide is gone and I missed the whole skeptics list. You'll have to forgive me. :( I did notice that the list of skeptics was longer than the list of advocates.

Are you a skeptic or an advocate of the deal?

Kevin likes Google's culture and admires how they've added to the working environment. He doesn't see people being thrown under the bus. Part of him says that that's a great thing and maybe life would be better if we could all join collectives. The other part of him sees Android, Chrome, and many more things pointing to one dominant source of information, and he views that as a bad thing.

Shelly says that personally he could care less if the deal goes through or not. But, speaking as a representative of advertisers who need to buy advertising, it's not a good thing because the mash-up will take the advertiser's ability to target the distinct differences of the two audiences. As a buyer and a planner he can't be effective. But Yahoo wouldn't do the deal if they didn't think they need to do the deal. Advertising has always competed to the death, but that's not done in technology because there's no reason to reinvent the wheel.

Jimmy says that Yahoo made a huge mistake by not being bought by Microsoft. It looks like this deal is the alternative to the failed Microsoft deal. To Yahoo this is a like giving up on something that should be a core part of their functionality. But his concerns for monopoly factors are that if you're big enough to be a player you don't want to sign everything off to the biggest guy.

Chris says that the deal has been suspended indefinitely. He also heard news today that Yahoo's and AOL's talks about merging have heated up. He doesn't think the Google Yahoo deal will ever occur.

James says that it makes him sad that some large center of innovation has folded. While Yahoo lost the edge years ago, the recent effect is that someone else out there who was figuring out a good way to build tools for advertisers is now lost.

Shelly says that a combined Yahoo AOL would be a content behemoth, but they still can't translate that value into wealth. To the level that Microsoft could buy that combined entity is funny. He also thinks it's funny that the evil empire talk used to be reserved for Microsoft.

Kevin says that Google built everything organically while Yahoo tried to build through acquisition. He thinks we're comparing apples to Buicks (hah!) because the two follow totally different strategies.

Is Google using their market power in organic search to propel YouTube, Google Maps and its other products in a disproportionate way?

Kevin says that Google's acquisitions came fewer and slower. We're going to see a lot more of search and behavioral targeting coming together because it's a means of collecting all kinds of information.

Shelly says that the practical thing is that some things require a certain scale. When the DOJ decided to break up Microsoft, not a whole lot changed because no one understood what a browser was, what an operating system was, or any of the basic terminology that was needed to talk about Microsoft.

Jimmy says that it only occurs to us to be shocked by the idea of the deal because it doesn't match what we understand about Google. Search is a report on the world, like a form of journalism. If people begin to sense that Google is promoting their properties over others, people would be suspect.

What is Google's responsibility to ensure accuracy?

James says that Google doesn't have any responsibility there. Shelly reiterates, saying that Google is a pipeline. It's like blaming radio waves for what's playing on the radio.

If Google knows everything about us, should we be more concerned about the aggregation of this data because of what the government could do with it?

Shelly says that it comes down to it, the electronic footprint that everyone leaves (completely apart from Google) is huge. Jimmy says that one of the interesting things about this is that if you're someone of intense interest, the government will be able to subpoena all that data, but for most people most of the time, it's not likely.

What does the Android phone do to the conversation of Googleopoly?

James says this is one of the best moves for openness in the telecom technology area. This is a great development in the mobile space because it's holding Apple's feet to the fire. Jimmy agrees and says it's good that there's something that will drive innovation.

Do you think the government is wise to look at Google for antitrust issues, or are there other places that they should be directing their attention that bear a far more negative impact on us as consumers and citizens?

Kevin says that they should look toward Google to figure out how to make money (hah!). James thinks we should worry about the ISPs first.

That was a fascinating conversation and I sometimes found myself entranced listening to the panelists instead of typing. That said, you can always check out alternative coverage. I saw Tamar Weinberg blogging away for Search Engine Roundtable, so if you like what you read so far, check it out!

Posted by Virginia Nussey on 10/ 7/08 at 4:26 PM | Comments (0)
See more entries in Google, Liveblog, SMX East 2008, Yahoo

August 20, 2008

Searcher Behavior Research Updates

And we're back from the fastest lunch I've ever eaten. Where does the time go? Moderating this session is Bill Muller (iProspect) and our panelists are John Marshall (Market Motive), Pavan Lee (Microsoft), Dr. Larry Cornett (Yahoo! Search) and Bill Barnes (Enquiro Search). I have to confess I just love the research sessions. Hard data just makes my little heart sing. Come on. You can't tell me you aren't excited about this one too.

I know you are because Bill is telling us about how every year this session totally fills the room it's in. Why? Because if you know more about the way searchers behave, you're going to be a better marketer.

John Marshall starts us off.

The interesting thing about search behavior is that it's not that difficult to get good data. The question is on Monday morning, do you understand search behavior? Most people turn to the keywords report in your analytics tool. That's a reasonable place to start but it's an extremely narrow view of the activity on the Web. You're only look at the keywords that brought people to your site. You only see the search results that brought people to your site. You're running into sample bias.

How can we really see the intent of people, not just the people who made it to your Web site. You don't want the whole forest view. You're probably not going to get the whole forest view anyway unless you have a Hitwise account or something. What you can look at is the single tree of your site.

The trick is to use the site search on your Web site. If you don't have site search, implement it. Even if it doesn't work, it's a great source of user intent. Search engine keywords only give you the people who came to your site. Site search gives you the intent of your users, conversion rate information. A lot of people ignore this data because it's free. Free data is often ignored. If you pay for something, you value it more.

Things that can go wrong:

• Mixed Case -- Google Analytics doesn't automatically change case for you so your data gets scattered across case. You need to convert it.
• Multiple results pages -- Some site search pages for 'no results found' don't get tracked by analytics. Make sure all the pages contain your analytics tracking.
• Usual JavaScript breakage
• Injected terms -- Most Web sites that have site search, they use it as a cheap landing page creation system. You have to filter that out of your data if you're doing that because it's not real data. No one is typing it in.

By using site search you're answering the question: what's the true intent of the users when they're on the site.

Site search data cannot replace competitive analysis but it's the cheapest way to get good data fast.

Pavan Lee is up next. She's from Shanghai.

Background from the New York SES: They've discovered that search listings have a branded value. Paid search listings have a stronger branding impact than organic search. There is a positive branding effect for both. They're trying to measure the brand lift.

They studied five brands in five spaces.

Methodology: Eyetracking and post-search survey.

Key findings: Search display and content ads are effective for branding stand alone but more effective together.

They asked "did you remember seeing an ad" 21% lift content 30% display. 38% both.

I can't see her slides at all.

In all cases with all questions, including lift in purchase intent, there was a brand lift and it was stronger for paired ads.

On the eye tracking side, search is still the most effective tool in attracting attention. There's a roll over impact on a multi-channel exposure. If you see search and display or search and content or search and display and content, it's more effective than just seeing any one of those.

Key takeaway: The power of three. There's a synergistic branding impact across content, display and search ads.

None of this data is public information.

Larry Cornett steps up to the podium.

His talk will build on John's presentation in a lot of ways.

Users do a lot before and after they're on the search page. He's going to talk about that, about the research they're doing, how users experience search, a little about crafting search and how they get from 'to do' to 'done'.

The reality is that the search page is just a tiny slice of online activity. Before the search, the user somehow comes to need to do a search. After the search, they want to go somewhere. They're going somewhere because they want to fulfill a task. The task is not getting to best buy. It's getting an iPhone. You need to know what happens after and how it all links back. How do you support them through the whole lifecycle of what they're trying to accomplish.

There is no single methodology that gives you the whole picture. Some ways that Yahoo does testing are:

• Search editorial
• Bucket testing
• Metrics & Analysis
• Search Science
• Focus Groups and surveys
• Eye-tracking research
• Ethnographic studies

How users experience search

• Starting context (what have they seen and experience before they query)
• Quick Scanning (one to three seconds)
• Information Scent
• Matching intent
• Quick Decisions
• Looking for answers (not a homework exercise. "Don't make me work")
• Feeling safe

They try to help crafting searches with 'search assist' (suggested searches). For most people search is hard. They're not experts.

Focus on the ultimate goal. They're looking to do something, they're wanting an answer. Yahoo SearchMonkey is an attempt at giving them that answer. It gives the user more information about what's behind the link and what's important to know.

What does this mean for marketers?

• Before the SERP
  • Starting context
  • The "Real task"
• On the SERP
  • Intent and information scent
  • Searchmonkey
• After the SERP
  • Fulfilling expectations
  • Being their "answer" and living up to the promise of the search result.

We thought that the reason people were having trouble with search was that it was an artificial session. But field studies showed us that the users were really having trouble formulating queries so we really tried to implement something that would help them.

Bill Barnes is the last to speak.

Their research is grounded in their search marketing and grew out of that.

Why is the first listing seen so important
Why do we scan in groups of 3 or 4
Why branding is important

[Standard heat map image, you've seen it a million times.]

They did experiments with the top SPONSORED listing and played around with really great ad copy versus just 'okay' ad copy.

When they did a survey they didn't ask about the listing, they asked about the search engine and if they'd use it again. The only difference was the ad copy but there was a huge lift in trust in the engine with the great copy.

Working memory: It's what comes to mind with recall. For some reason, we're hardwired to think in threes or fours.

[Oh no, my battery is dying]

There's a 16 percent increase in brand association when brand is the Top Sponsored and Top Organic Results. On an unbranded query. The really interesting thing is that the recognition of OTHER brands drops away at the same time.

There's an 8 percent lift in brand purchase. If you're not there, you lose 16 percent brand lift.

Even for branded queries, you get a brand lift if you appear. Should you buy your branded terms? Yes.

Eyetracking finding: Brand fixation only occurs in the TITLE and the URL not in the description.

If you're a familiar brand to the searcher, they will often skip the sponsored listings at the top. If you're buying the top sponsored, write your copy for a NEW user.

If you have brand A and brand B in sponsored with Brand A in top organic, brand A gets a HUGE lift.

Key Findings:

• INTENT is the most important thing
• Organic and sponsored combined give the biggest brand lift.
• Be aware of who else is on the serp
• Write your ad copy to new clients.
• Don't assume your brand will be in the consideration set. If you're not on the page, you're forgotten.

Q&A

The first question is does offline affect offline. The answer is yes, though the panelists don't say that. Go read the Re Search Online, Purchase Offline session from yesterday.

Why do search views get longer?

Pavan thinks it's because searchers are looking for something in particular whereas display and content ads are push forms of advertising.

Do they really only spend 1-3 seconds and how often do they click?

Larry: It's on average. In some cases, for navigational queries, that's less than a second. It might be longer at home but yeah, it's amazingly fast.

Bill: Females look longer and shop around, males just go straight to results. There's a free paper available.

Pavan: Search intentions lead to searcher behavior. Fact based search stays organic. Commercial searches tend to be more broad. It also varies by culture. Chinese spend twice as long as Americans.

John mentions that his contention is that the site search queries are the same queries that are being typed into the search engines but they're just not getting to your site.

Is the suggested search condensing the search queries?

Larry: Yes. People are moving to longer queries and that search assist does jump them to the queries that will get them to the answer faster.

Are there differences in lift by categories?

Pavan: Yes there is a difference in lift across different verticals but in all cases it does result in lift.

[Long set up about pretending to be a confused searcher and poor SERPS] What can be done to help confused searchers?

Larry: Search assist is just one way. It works mostly for shorter queries?

Does the golden triangle change with non-roman character sets?

Pavan: In Chinese, the scanning pattern is very different. It's a rectangle. You have to look at everything to put together meaning.

Would you suggest not trying to dominate the organic?

Bill: No, never. Always optimize.

What plays into search assist? How does it affect PPC?

Larry: Nothing is paid in those.

John: The hidden message there is: No you can't spam the suggestions.

John says that the other thing site search is good for is manifesting usability problems and for doing competitive intelligence.

If you rank 1 on a non branded term, should you also be number one in paid search as well?

Bill: That's exactly what our research showed. That said, always test and retest and see if the ROI is worth it. Clicks went 50/50 on paid and organic, so make sure that you're testing and monitoring.

Posted by Susan Esparza on 08/20/08 at 2:10 PM | Comments (0)
See more entries in Analytics, Live Search, Liveblog, Pay Per Click / Online Ads, SEO Tips & Tricks, SES San Jose 2008, Search Engine Optimization, Yahoo

July 9, 2008

SEO Headlines

Google's New Keyword Research Tool

Our PPC guru Nick Guastella was all aflutter this morning over the news that Google has added search counts to its keyword research tool, thereby turning on another light for SEOs and search marketers. According to Google:

When you use the Keyword Tool to search for relevant keywords to include in your keyword list, you'll be able to see the approximate number of search queries matching your keywords that were performed on Google and the search network. These approximate numbers are intended to provide better insight into keywords' monthly and average search volumes than previously provided by the tool.

The new data will help search marketers with keyword choices, spotting trends, budget planning and will help them outline their account structure. I supposed props go to Google for being so aggressive about pushing out new tools to give search marketers more data, more numbers and more insight into optimizing their campaigns. I'm going to do my best to avoid the conspiracy theories and be excited about the new data. Nick seemed to be.

For more details, Google has an extensive guide to its Keyword Tool. I don't think many are still mourning the loss of Overturn's keyword research tool.

But Srsly, WTH is Lively?

Yesterday afternoon word started to spread about Google's new virtual world nicknamed Lively. It's a browser-based virtual environment that will tie in to social networks like Facebook, OpenSocial, and MySpace. Okay. The whole think smells of Orkut. It's just another social networking attempt that Google's audience never asked for, doesn't want, and will likely never use (or at least not here in the States). I'm not impressed; in fact, I'm confused as to why they even bothered.

It feels like not even Google knows what it wants to do here. As GigaOm notes, in a recent Virtual World News article Google's Head of 3D Operations (I love that a 3D division even exists) Mel Guymon makes it sound like they're only in the virtual space because it seems like that's the place to be. That's a great way to induce Product Fail. The obvious assumption would be that Google developed Lively as a way to (in time) get users to generate content that they can then place ads on. Lively has integration with Google products like YouTube and Picasa so that may be another way to generate more clicks and ads, but that's not nearly enough to make it exciting.

It seems that if you're going to release something like this, it darn well better be superior to its nearest competitor. In this case, though the missions are different, that's Second Life. And Lively's not even on the same wavelength as SL. People in virtual worlds demand complete control of their surroundings and freedom to explore. Lively fails to offer that. So where's the incentive to switch? There isn't one.

Google, I know you're all excited about another chance at monetizing something, but next time try and do a better job of masking it behind something that's maybe useful.

adCenter Makes Impressive Strides Against Y!SM

Barry reports on the buzz that Microsoft's adCenter seems to be on the rise much to Yahoo's dismay, with advertisers reporting more spend on adCenter than with Yahoo Search Marketing. Over at Search Engine Watch one member noted that adCenter as outperformed Yahoo in both conversions and CPL over the last month. At Sphinn, Kate Morris argues the same. Barry says the "tide is turning". Is he right?

I certainly hope so. I'm not a fan of much of what Microsoft puts out there, but adCenter has long been touted as the superior platform despite its pea-sized traffic. Maybe with search marketers starting to see rewards, they'll be more likely to increase their spend over there. But if they do, it doesn't curb adCenter's major hurdle - the fact that the audience isn't there. And the audience isn't there because the Live search engine is....nowhere near where it should be. Maybe the folks in Redmond could stop bullying Yahoo and get on that. They just may have something here.

Fun Finds

I'm a huge fan of Patrick Winfield's recent article The 10 Best Ways to Find the Perfect Image for Your Blog Post. Some seriously good stuff in there.

Posted by Lisa Barone on 07/ 9/08 at 4:29 PM | Comments (2)
See more entries in Google, Microsoft, Pay Per Click / Online Ads, Yahoo

June 19, 2008

SEO Headlines

According to Yahoo weather, it was 105 degrees earlier in Simi Valley, CA. The palm trees and ocean views don't look so inviting right now, do they? Yeah, I didn't think so. Can I come live wherever you are?

Yahoo Opens Up YMail. Or At Least To Some

For reasons I don't quite understand, Yahoo began offering email addresses under two new domains: rocketmail.com and ymail.com. I guess that's nice. Though I tried to register a ymail email address and Yahoo said I had to wait. I guess they're rolling out the feature slowly and I'm not yet on the invite list. Don't these people know who I am? How am I supposed to secure thelisa [at] ymail [dot] com if they let everyone sign up before me?

But enough about me.

Yahoo says the goal behind the new email accounts is to "attract Web surfers unhappy with their current addresses". They argue that there are plenty of people out there who feel "stuck" with an "ugly" email address. Yahoo's here to help! Or at least they're pretending that's why they're here. They're really just shouting "ymail is kind of like gmail, right? It sounds the same. Use us! OUR EXECS ARE LEAVING!" Besides sounding a bit desperate, I just don't think it's a good idea. Why fragment your brand like that? And are there really that many people feeling slighted over their email address that they need to create a brand new domain and give life to an old one? Do they realize what a pain it is to switch email addresses in the first place? Who wants to invite that amount of misery on themselves? [Tell me about it. Four years and I still can't get my relatives to stop emailing my hotmail account. --Susan] Just because we're not talking about me anymore doesn't mean we're going to magically start talking about you. No one even likes you. Don't pretend you get email.

Icahn Issues The Ichan Report

Everyone's favorite thorn in Yahoo's side Carl Icahn has launched his own blog to (hopefully) continue to further torment Yahoo with! If only because it's kind of fun to watch Yahoo squirm.

According to the New York Times, Icahn's blog looks to "address the lack of accountability on the part of corporate boards and the myth of shareholder". Sadly, however, it doesn't look like we're going to hear any behind the scenes Yahoo/Microsoft dirt. We're not even going to hear him take Jerry Yang to task. Bummer.

C'mon, Carl. You can't tease us! If you're going to start a blog, give us something entertaining to read. None of this veiled and abstract stuff you're doing. We know you want to bring Yahoo down and get it sold to Microsoft. Use your new microphone.

[grabs popcorn, waits.]

Fun Finds

Jeremy Zawodny announced that he'll be heading to Craigslist in July. Does that not seem like the most random job shuffle ever?

Barry Schwartz confirms that page load time now is a real factor in Google's AdWords quality score.

Michael Arrington calls out the A.P. for quoting 22 words of his content. He says they now owe him $12.50. Hee. Sometimes don't you just love Arrington?

Posted by Lisa Barone on 06/19/08 at 5:16 PM | Comments (2)
See more entries in Yahoo

June 12, 2008

SEO Headlines

Should Your Employees Be Your BFFs?

Barry Schwartz pointed me to an article from Ad Age this morning that asks Do Bossfriends Create Great Employees. It's funny, because if you asked me that a year ago my answer would have been completely different than what it is now. When I was still wet behind the ears I would have said that a boss can absolutely be buddies with their employees. I may have even rattled on about how it creates team unity and that hitting a beer bong in a social setting with those whose paychecks you sign was totally okay. Now, however, that kind of behavior would make me cringe.

As a boss you should be friendly with your staff. You should create an environment of trust and openness and free expression, but that doesn't mean you should be best friend. There's absolutely a line you have to draw. You don't want to get into a situation where employees' feelings are hurt because their bossfriend suddenly has to care about getting work done or the bottom line. You don't want to open up an environment where employees get too comfortable or too casual and their professionalism suffers. You want there to be chemistry and for your employees to feel supported. That doesn't mean that you have to help them close down the bar.

I think we manage to have a good balance around here. We have a pretty casual environment and open door policy when it comes to getting time with Bruce. We have company BBQs, bowling days, movie nights, spontaneous lunches, etc. But I'm not hanging out at Bruce's house tonight to watch the Celtics beat the Lakers. I'll be doing that at home, with people who are actually my friends.

Sorry, Bruce, but I just want to be friendly. I don't want to be your BFF. :)

Yahoo, Microsoft, Google & The Internetz

Yahoo says that their talks with Microsoft really are over and that no deal has or will be made. Take a moment to compose yourself. It'll be okay. Here's a tissue.

Better?

Yes, TechCrunch reports that though Microsoft and Yahoo had both gone back to the table for negotiations, in the end Microsoft couldn't justify the price Yahoo was after, and Yahoo realized it'd just be neutering its search engine anyway. The two will just have to go it alone.

Don't worry, Yahoo's not about to fall out of the headlines. Just this afternoon they announced a non exclusive search agreement with Google like we all thought they were going to. The deal will allow Yahoo to run Google ads alongside its search results and some of its Web properties in the United States and Canada.

From the press release:

"Under the terms of the agreement, Yahoo! will select the search term queries for which - and the pages on which - Yahoo! may offer Google paid search results. Yahoo! will define its users' experience and will determine the number and placement of the results provided by Google and the mix of paid results provided by Panama, Google or other providers. The agreement applies to paid search and content match and does not apply to algorithmic search. The agreement also applies to current partners in Yahoo's publisher network."

Companies With An Identity Crisis Fail

Dave Goldenberg had a great article over at Digital Web Magazine entitled Why Do Web Startups Die? Lack of Alphalpha. Dave says the biggest reason new companies fail isn't because they don't have the talent or the drive to back up what they're trying to do, it's because they never really identify what it is they want to do. They don't know who they are, where they're going, what makes them different, etc, and as a result they fall on their face.

I mentioned this in Tuesday's Avoiding Product Fail post, but you really have to know what you're creating and what niche your product is going to fill before you launch it. Otherwise you those vital first moments of your existence solving your own identity crisis when you should be out presenting a strong brand image and getting your message out to your audience. As was hailed in the Cre8asite Forums, you don't want to be that nerd in the singles bar. You want to assured, nimble, and ready to woo the masses.

It's hard enough to launch a new company. If you don't have a clear plan for how you're going to do it and become the next TechCrunch, you may as well not even try.

Fun Finds

Christopher Hart, the new Director of Regional Operations for Bruce Clay East, sat down with Jim Hedger during last week's SMX and talked about his plans for the new office, bringing SEO training to the East Coast and how BC East will fight through the noise of New York.

Chris Brogan tells us all how to be sexier in person. C'mon, you know you were googling it last night.

Posted by Lisa Barone on 06/12/08 at 4:36 PM | Comments (2)
See more entries in Google, Search Engine Optimization, Search Engines, Yahoo

June 4, 2008

Search Friendly Development

Good morning, Friends. It's time for Day 2 and I'm coming to you live from the brand new Developer Track. The Developer Track is awesome because it comes complete with a fancy waterfront view. I'd also mention the yummy bagel I'm eating but I think Michael VanDeMar is going to come and kick me if I do. He's over my bagel stories.

Vanessa Fox will moderate as speakers Nathan Buggia (Microsoft), Maile Ohye (Google) and Sharad Verma (Yahoo) get us started.

Up first is Nathan Buggia. He says he rewrote his entire presentation last night based on what people were talking about on Day 1.

Microsoft is working on lots of big, hard problems. Stuff like:

• Affiliate tracking
• Session management
• Rich Internet application
• Duplicate content
• Geo-location
• Understanding analytics
• Redirection
• Error Management

Advanced search engine optimization is analytics. That's what differentiates it from regular search engine optimization. It means you're at a larger company with more resources (um, not necessarily). Implement things in a logical order. See what the impact is on your customers and the engines and decide if that's the right thing to go forward with. Do not implement something because you heard someone on a panel say it was a good idea. PageRank sculpting is a good example of that. Everything on the Web is an opportunity cost.

Nathan says to watch out for complexity. If you build cloaking or situational redirects into your Web site, you can add a lot of complexity to your site. It becomes hard to notice if you have problems on your site because stuff is hidden from even you. You want the simplest architecture you can have. Microsoft says cloaking isn't all bad, but it's never the first, second or third solution they recommend.

All Web sites have the same first problems. The first problem is accessibility. That's where people should start. Can a crawler get to your Web site? Are they hitting 404s? Do you use Flash or Silverlight and are they monopolizing the user experience? Take a look at canonicalization. Are you dividing all your PageRank and reputation?

Search engines are always changing. Someone can come up on the stage and claim they have the big new tactic for search engine optimization and then that may change in a year. What is consistent are the Webmaster Guidelines. Those are things that in spirit all the search engines agree with. If you go to Google's Webmaster Guidelines and adhere to the spirit of them, then you're working with the search engines instead of against them.

Nathan gives us an example and uses Nike.com. Nike is a brilliant company. There are few companies that can do the type of branding that they did with Just Do It.

When you go to Nike.com you see the Flash loading. Then you select language, region, etc. Then you get another loading screen because they're going to play a full minute video. It takes eight seconds to get to that video. Maybe people don't have eight seconds. Maybe they only have one second. The second run experience is 3 seconds because of the cookie Nike puts on your computer. The cookie resets every day. If you are blind or ADHD, you have a really bad experience on that site.

The site also isn't great for search. He shows us the HTML behind the page. There's no Title tag. There's nothing. It's just a Flash application. Basically they're cloaking. The site is also really complicated. Nike has over 2 million pages on their Web site and they're cloaking for a lot of them. He shows what the Nike SERP description was for a few days after their cloaking broke. It was a user error.

Every investment you make is another investment that you can't make. If you're investing all in cloaking, there are other people out there NOT investing in those things. If you type in [Lebron James shoes], Nike doesn't come up.

Alternate Implementation

Throw your rich object at the top of the page and then use JavaScript at the bottom to detect what the div does. (If I mangled that, please feel free to correct me in the comments. As awesome as Nathan is, I don't speak tech geek.)

Advanced search engine optimization is not spam.
Search engine optimization does equal good Web design.
Design for your customers, be smart about robots and you'll enjoy long-lasting success.

Sharad Verma is up.

Sharad says he loves his job. This is an opportunity to serve his customers. When he's not working he loves to travel. Last week he was in Machu Picchu, Peru. He's giving us a bit of a history lesson and telling us how he took trains and buses on his journey. I'm not sure where this I going but it will tie together soon. Oh, I get it. The moral of the story is that Machu Picchu is accessible and easily discovered. I see what he did there.

As a site owner you're serving both your users and robots. You need to design your site so you're not alienating either of them. There are three cranks behind the box - crawling, indexing and ranking. You have control over all three, but more control over crawling.

How do Spiders Crawl Your Web site?

They start with the URL, download the Web page, extract links from the Web page and then follow more links. Sometimes they find invisible links or sometimes they see links but decide not to crawl the content. That could be because the links are excluded in your robots.txt or because they're duplicate links.

Search engines find your contact via the organic inclusion from crawling. All you have to do as a site owner is put up your site, get links, and let the crawlers in. They'll do the magic. If you're not satisfied with what they're crawling, then you can supplement that with feeds.

Roadblocks of Organic Crawl

Search engines do not understand JavaScript. They're starting to understand it but they're far away from being able to full crawl it. He recommends turning off your JavaScript and seeing if you can navigate your Web site. Is all the content reachable?

Flash: Make sure your site can be read by a robot. If you're using Flash, make sure you're offering up alternative navigation.

Dynamic URLs: Difficult to read, lead to duplicate content, waste crawl bandwidth, split the link juice and are less likely to be crawled and indexed.

Best Practices:

• Create user friendly, human readable URLs
• 301 redirect dynamic URLs to static versions
• Limit the number of parameters
• Rewrite dynamic URLs through Yahoo! Site Explorer

He asks how many people use Site Explorer and their Dynamic URL Feature. Log in and authenticate your Web site. It allows you to remove parameters from URLs.

Duplicate Content

Consequences of duplicate content: Less effective crawl, less likely to attract links from duplicate pages.
Solutions to duplicate content: 301 duplicate content to the canonical version, disallow duplicate content in Robots.txt

Other Best Practices:

• Flatten your folder structure
• Redirect old pages to the corresponding new pages with 301/302
• Use keywords in URLs
• Use sub-domains ONLY when appropriate
• Remove the file extension from the URL if you can
• Consistently use canonical URLS for internal linking
• Promote your critical content close to the home page

You can also get your content included through feed based crawling. You can provide feeds through their Sitemaps Protocol to tell the crawler were to find all the pages on your site, especially your deep content. Sharad recommends using all the Meta data supported by Sitemaps Protocol.

Do not exclude your CSS content in the Robots Exclusion Protocol because the engines want to see the layout of your page.

Search engines want your content. Break down those accessibility barriers and let them do their job.

Maile Ohye is up last.

Google wants to help users create better sites. If you have better sites, we all have a better Internet. Aw. She's going to tell us how to enhance your site at every stage of the pipeline. Maile talks like an infomercial.

Crawlable Architecture

Consider progressive enhancement. This means you don't just begin with Flash. You start with static HTML and then add the "fancy bonuses" like Flash and AJAX later. Then the fancy stuff becomes a complement to your Web site instead of your entire site.

She looks at a page/site that's rich in media with HTML content and navigation - the Dramatic Chipmunk video on YouTube. The video is in Flash, but there's descriptive content on the page (title, description, user generated content in the comments) and HTML navigation.

Consider sIFR for Flash

JavaScript detects if Flash is in installed.
With No Flash, it displays the regular text. With Flash on, you get the Flash.

If you do that the text must match the content viewed by enabled users. It must be accessible to screen readers and search engines.

Consider Hijax for AjAX

Format JavaScript with a static URL as well as a JavaScript function. She gives us a long URL and says that the search engines often ignore fragment (#f00=32) but respect parameter (?foo=32). I'm hoping that makes sense without you having to see the long URL.

Google Webmaster Central

Webmaster Tools: They give crawl errors if you verify your site. In crawl errors, be sure that what you see is what you expect. They'll show URLs blocked by robots.txt, make sure that's what you want. They'll also tell you about time out errors and unreachable links. Use it to verify your link structure and that all your links are findable.

Promote your quality content. Set preferred domain to www or non-www. You don't want to run two versions of your Web site. [As a note, this doesn't always fix the problem. Be consistent in your linking and don't rely on Google to do your work for you.--Susan]

To reduce duplicate content, keep URLs as clean as possible, internally link to your preferred version and store visitor information in cookies then 301 to canonical version.

Use a cookie to set the affiliate ID and trackingID values.

Proper Use of Response Codes

Use 301s for permanent redirects.
Signals search engines to transfer the properties like link popularity to the target URL. This applies to situations like moving a site to a new domain and modifying the URL structure.

Anatomy of a Search Result

Create a unique, informative title. It acts as informative signal of the URLs contents to a search engine and user. You don't want your title to say "Untitled". She talks about how Webmaster Tool can help you locate Title tag issues.

Snippets: Provide the user more content about each search results. The quality of your snippet can impact your click-through.

Influence snippets with Meta Description. Meta Descriptions can be utilized by Google in search results. Meta keywords are of low priority.

Final thoughts from Maile:

• Verify Crawl errors as expected
• Creative descriptive titles, consider adding useful meta descriptions
• Submit site maps for your canonical URL
• View Webmaster Central blog posts

Posted by Lisa Barone on 06/ 4/08 at 10:18 AM | Comments (0)
See more entries in Design, Google, Liveblog, Microsoft, SMX Advanced 2008, Search Engine Optimization, Search Engines, Yahoo

June 3, 2008

Organic Track: Bot Herding

Back from lunch and Rand Fishkin is moderating a star studded panel. We have Adam Audette (AudettaMedia), Hamlet Batista (Nemedia S.A.), Nathan Buggia (Live Search Webmaster Center), Priyank Garg (Yahoo Search), Michael Gray (Atlas Web Service), Evan Roseman (Google) and Stephan Spencer (Netconcepts).

[Ooo, we're rocking the Tears for Fears. It's pretty sweet. SMX always pulls out the good stuff. I'm really loving the Organic room this time around. The first row is literally two feet from the stage. I could totally throw my mini bottle of water and hit Adam Audette square in the face. Not that I would. I really like Adam. Maybe I'll take out Rand instead?

Holy Jesus. I almost just kicked over the big projector that the speakers watch. Blogger Fail.]

Rand gets this whole show going. He says he's moderating. He's wearing a tie. I don't think I've ever seen Rand wear a tie. Up first is Michael Gray.

Michael says that when you first buy a house you're poor. Over time you make more money and can afford to air condition your house. But no matter how much money you have you'll never air condition your mail box because you have a better use for your money.

Similarly, you have a Web site. You don't have a lot of links. You don't have a lot of Page Rank. You're not going to send that PR to your Contact page because it's not a good use of your resources. You want to send your PR to the pages that make the most sense for you. You want to send it to the places that will give you the most sales and the most leads.

PageRank and link equity, how much do you have: Many Web sites, especially smaller or new Web sites, don't have a lot of PR. They have to use and maximize what little they have and direct it to the right places that make the most difference.

Deciding What to Sculpt Out

Who wants to rank for privacy policy, terms of use or contact us?

Locations: Unless you are multi-location business, put your address in the footer and sculpt out the Locations page.

Company Bios: Unless you are involved in reputation management scandal, sculpt them out.

Site-wide footer links, advertising stats, rates and legal pages.

How to Sculpt:

• Nofollow: Quick and easy, but may be a signal to search engines that an SEO or advanced webmaster is involved.
• JavaScript: Old school, relies on client side technology, currently bots don't crawl it but this may change in the future.
• Form pages, jump pages, redirect pages - More complex to implement and maintain. Search engines currently don't follow them but that may change

Be Consistent. If you're going to nofollow something, do it with all of your links and then do it in your robots.txt. Don't block them one way and then allow them in another. This will account for outside links and any spider or search engine quirks. He says that he's seen most benefits on mid-level sites where they sculpted out blocks of 20-50 non-conversion based pages.

Do it now or wait for a rainy day? He says do it now. If you have any critical or serious issues this can take a backseat. Otherwise, unless you have a large or very complex site, PageRank sculpting is a 1-2 day project at the most for any CMS or template based site. It's easier to get it right now than to get back and fix after you launch.

Adam Audette is up to give us 8 arguments against sculpting PageRank with nofollow. He used to do it but he's now starting to slow it down. They use it far, far less.

More Control: Have a mechanism at the link level to control spider behavior is good. However, we don't know enough. We don't know how much PR we have on a domain. We don't know how much we have on a page or how much a link takes off a page. We're attempting to control the flow of internal PR but we don't know how much we have. We don't know how much it fluctuates. It's imprecise. It's like using a precise surgical tool while blindfolded.

It's a Distraction: There are a lot of things we can do to make our sites better. Matt Cutts has said that sculpting with nofollow is a second order effect. It can also mask other issues - focus of a page, keyword dilution, user experience, etc.

Management Headaches: When you have a large site you may have many departments working on a page. What rules are in place? It's confusing. Why are 5 links nofollow'd on this page? How do you preserve it?

It's a Band-Aid: People are using it to try and address a symptom they're seeing on a site. They're not taking care of the problem.

Where's the User?: Think of a site with tons of PageRank feeding that into mediocre places, thereby raising those pages in the SERPs. Are we giving more power to high authority domains?

Open to Abuse: Every tool is open to abuse, but you can think of all kinds of creative ways to use nofollow. When will nofollow start being abused and how will the search engines react? Matt Cutts says it's okay, but there are good and bad ways to use a technique. He may look at your site and think you're using it in a bad way.

Too Focused on Search Engines: Advanced search engine optimization has always been about what's right for your users and what's right for search engines. Too much use of the nofollow puts too much focus on search (specifically on Google). Does this help your users? Would you do this if the search engines didn't exist?

There's No Standard: There are multiple definitions for nofollow and each engine may treat it differently. Nofollow started for blog comment spam. Then it went to paid links. Now it's to control your internal PR. What's it going to be next? It moves too much.

[Rand shares a detail but says you can't ask him how he knows it. He says 5 percent of pages on the Web currently have a nofollow'd link on them and 85+ percent are using it internally.]

Stephan Spencer is up.

Duplicate content is rampant on blogs. Herd bots to permalink URLs and lead in everywhere else. (Archives by date, category pages, tag pages, home page, etc). You can use optional excerpts to mitigate that a bit. [Whatever that means.] It requires you to revise your Main Index Template theme file.

Stephan says to include a signature link at bottom of your post/article. Link to original article/post permalink.

On ecommerce sites, duplicate content is rampant because of manufacturer-provided product descriptions, inconsistent order of query string parameters, guided navigation, pagination within categories, tracking parameters, etc. Selectively append tracking codes for humans with white hat cloaking or use JavaScript to append the codes.

Pagination not only creates many pages that share the same keyword theme, but it also creates very large categories with product listings not getting crawled. Thus lowered product page indexation. Do you herd bots through keyword-rich subcategory links or View All links or both? How to display numbered links? You have to test because your mileage will vary.

PageRank Leakage: If you're using Robots.txt Disallow, you're probably leaking PageRank. Robots.txt Disallow and Meta Roberts Noindex both accumulate and pass PageRank.

Stephan talks about the magic of regular expressions/pattern matching and I'm not even going to pretend that I followed any of it.

Some expressions I did manage to catch:

Mod_rewrite specifics
Proxy page using P flat
QSA flag is for when you don't want query string parameters dropped.
L flag saves you on server processing

Got a huge pile of rewrites? Use rewritemap

He talks about conditional redirects. It's way black hat. I'm covering my eyes.

Error Pages: Drop them out of the index by returning a 200 status code instead so that the spiders follow the links. Then include a Meta robots no-index so the error page itself doesn't get indexed. Or do a 301 redirect to something valuable and dynamically include a small error notice.

Hamlet Batista is up to talk about white hat cloaking.

Good vs. Bad cloaking is all about your intention. Always weigh the risks versus the rewards of cloaking. Ask permission - or just don't call it cloaking. Don't call it cloaking. Call it IP Delivery.

When is it practical to cloak?

The main idea of cloaking is about making more of your content accessible to the search engines. Parts of that can be because you're using a search unfriendly CMS, if you have content behind forms or if you're a rich media site. It can be that you're a membership site (free vs. paid). He's also going to talk about using it for site structure improvements, geolocation/IP delivery, and multivariate testing.

Practical Scenario 1: Proprietary Web site management systems that are not search-engine friendly.

Regular users see URLs with many dynamic parameters, but the search engines see friendly URLs. Your users will see URLs with session IDs, but with simple cloaking the search engines see URLs without session IDs. Your users will see URLs with canonicalization issues. The search engines see URLs with consistent naming convention. Your users see missing Titles and Meta Descriptions. The search engines see automatically generated tiles and Meta Descriptions.

Practical Scenario 2: Sites built in Flash, Silverlight or any other rich media technology.

With cloaking, you can give users a completely Flash site and the search engines will see a text representation of the graphical, motion and audio elements.

Practical Scenario 3: Membership sites.

Search users see a snippet of premium content on the SERPs and when they land on the site they are faced with a reg form. Members see the same content the search engine spiders see.

Practical Scenario 4: Sites requiring massive site structure changes to improve index penetration.

Regular users follow the structure designed for ease of navigation. Search engine robots follow a link structure designed for ease of crawling and deeper index penetration of the most important content.

Practical Scenario 5: Geotargeting

Practical Scenario 6: Split testing organic search landing pages.

How do we cloak? In order to cloak you have to ID the robot and then deliver the content. You can do that via a few methods:

• Robot detection by HTTP cookie test.
• Robot detection by IP address
• Robot detection by double DNS check
• Robot detection by visitor behavior

Hamlet runs out of time and Rand nearly yanks him off the stage. Poor Hamlet. He didn't get to finish his presentation, but Rand was just doing this job.

Priyank Garg is up.

Robot Exclusion Protocol: Allows publishers to tell Robots access permissions for their content.

Robots.txt: Introduced in '90s. Defacto standard followed by all major search engines. Allow site level directives for access to content.

META Tags: Page level tags. Allow finer controls.

What is the standard? Does everyone work the same? Priyank says the engines are working together to make standards across all engines. The engines all support page level tags like HTML Meta, noindex, nofollow, nosnipper, no archive, noopd.

They want to have all the engines come out with this at the same time so there is no confusion.

In the Q&A, Evan Roseman says they don't view uses of nofollow as some type of "flag" for SEOs. They're standard of nofollow has not changed over the years. People have simply begun using it in new ways.

Nathan Buggia says a nofollow'd link is viewed as any other link. MSN Live does not support nofollow. [Update: Nathan retracts his statement later on, much to the disappointment of bloggers everywhere.]

Posted by Lisa Barone on 06/ 3/08 at 3:00 PM | Comments (1)
See more entries in Google, Liveblog, Microsoft, SEM Events, SEO Tips & Tricks, SMX Advanced 2008, Search Engine Optimization, Search Engines, Yahoo

May 28, 2008

SEO Headlines

The Belgians Are Back Bothering Google!

I received quite a treat this morning when I was listening to the DailySearchCast and heard Danny mention that those funny Belgians are back in the spotlight and causing trouble for Google. You may remember that I've been rather vocal about how stupid I find this whole situation to be. If you don't remember, here's the breakdown: A group of Belgian newspapers sued Google for copyright infringement because they (the Belgians) were too lazy to use a robots.txt file, which landed their articles in Google News. They didn't seem to like that. In some crazy universe where fair use does not exist, the Belgians actually won their lawsuit and Google had to place a ridiculous note on their home page and remove the content. And now the Belgians are back!

Since we last heard from them, Google appealed the judgment and tried to negotiate with the Belgian newspapers outside of court. Sadly, the negotiations weren't going as quickly as the newspapers would have liked and Google began referencing the papers again. Now the Belgian newspapers are asking the courts to award them $77 million in damages. Seventy-seven million dollars!

I'm sorry, but I'm still inclined to file this away as the Most Ridiculous Lawsuit Ever. I know we have commenters who like to come and correct me each time I mention the Belgians and their idiocy, but I haven't been swayed. And this new twist to the story just adds to its lunacy. As Danny joked on the SearchCast this morning, I doubt that the Belgian newspapers have made a combined $77 million since Google News was born, so that seems a bit hard to claim that Google has cost them that much. We'll see what happens. All I know is that Google vs. The Crazy Belgians are what good blog entries are made of.

CW: Watch This Show Online, That One Offline

In stupid company news, after taking Gossip Girl offline due to too many viewers, the CW is now looking to create "cwingers". What in God's name is a cwinger? It's an ad-supported video clip that lives half online/half offline. Basically, viewers will get to see a short video inside their favorite CW show, and then they'll have to go online to get the next installment before the conclusion airs on TV. [...huh? --Susan] It's three parts. The first airs on television, the second airs on the Internet and the third airs on television. See how confusing?

Um, hi, as a loyal CW television, you're completely confusing me. What do you want me to do? How do you want me to consume your programming?

The CW is sending a mixed message with the way they're handling online video. You either embrace it or you don't. I don't think you can pull your flagship show offline one week, and then decide to create a whole new "cwinger" format the next. Maybe I'm wrong. The CW has been pretty nonconventional with the way they've done programming in the past, so it may just work.

Horrible name aside (Cwingers? Is this a child porn ring?), I think the CW is going to confuse viewers. You're telling them not to watch X online but to remember to tune into Y and Z so they can see how the story unfolds? Pick a brand message and stick with it.

Yahoo Will Soon Announce A Deal With...Someone

BusinessWeek says that amidst pressure from shareholders, Yahoo will work out a deal with Microsoft...or Google. Yup, the deal is so close that Yahoo doesn't even know who it will be with yet. Either way, BusinessWeek says that "something" will definitely happen soon. Right. Like my head will explode from all the baseless speculation.

Oh, the stupidity headache.

Fun Finds

Andy Beal gives us all a reminder that SEM Scholarship entries will start appearing Marketing Pilgrim today, so make sure you keep your eyes open for the next big SEO/SEM superstar.

Wired's founding editor Louis Rossetto writes a letter to his sons and recalls the dawn of the digital revolution.

Do WiFi allergies exist? [No. --Susan]

Posted by Lisa Barone on 05/28/08 at 2:04 PM | Comments (1)
See more entries in Branding, Microsoft, Search Engine Optimization, Search Engines, Yahoo

May 19, 2008

SEO Weekend Update

Microsoft, Yahoo Continue To Annoy Me

I didn't really believe that all the MicroHoo chatter was behind us, but part of me really, really hoped. I'm a little tired of talking about nonevents, but here we are again. On Sunday, Microsoft issued another weekend statement saying that they're "continuing to explore and pursue...an alternative that would involve a transaction with Yahoo! but not an acquisition of all of Yahoo!" Er, what does that even mean? Epic Yawn, Microsoft.

No one really knows what Microsoft's cryptic note refer to, but we're all guessing anyway. The most popular theory is that Microsoft is looking to get its hands on some of Yahoo's search share. Sounds viable. Personally, I think the way Yahoo can best help Microsoft is to bring some eyeballs to Microsoft adCenter, which most agree is the best ad serving platform that no one's using. I'd love to see that platform get some actual traffic.

Regardless of what Microsoft's latest sonnet really means, Yahoo appears to be back the bargaining table, which I'm sure has nothing to do with the fact that its own stakeholders were revolting. Fun!

Should Brands Buy Back Their Fan Pages On Facebook?

There's a post over on The Unofficial Facebook Guide that tells the sad story of users getting banned for creating branded fan pages. Nick O'Neill writes that ever since Facebook unveiled Fan pages onto the site, enthused brand evangelists have been jumping at the chance to create pages for the brands they love. Sadly, it seems the pages they've created are being taken down and, in some cases the users' accounts are being banned. Yikes.

While banning personal accounts because a user was excited about a feature you offer is 100 percent ludicrous, I do think it's a good idea for brands to take control of their Facebook pages, even if John Battelle doesn't. Sure, it's likely that these brand evangelists are completely well-intentioned but I don't think you should hand over your logo, your message, or your public face to someone simply because they were first to try and register the page. You also don't want a hundred splinter Fan pages sprouting up for your brand, you want one official one where all of your fans can unite and support you. And if your Fan page is going to be in front of that many eyeballs, you want to make sure you're the one in control of it.

Remember our friend Jackie Liebergott from last week? On Thursday I wrote that My College President Kills Kittens and showed how someone had created a false Facebook profile for Jackie Liebergott, the president of my alma mater Emerson College. Well, it turns out they've also created a fake Jackie Liebergott Fan page. Now, if you were Emerson College, wouldn't you want to get control of that page? Poor kitten-killer Liebergott.

Encourage users to engage, to write wall posts, to answer polls, to go out in the real world and evangelize, but you have to hold on to the keys. Otherwise you're just opening yourself up for disaster.

Would Yahoo Be Stronger Without Search?

At Search Engine Journal, Loren Baker asks if Yahoo would be stronger without search. It's an interesting theory but not one I'm inclined to support. Perhaps Loren's right in that Yahoo would become more profitable, but it's still not a direction I want to see them take. They might make more money but would you still respect them in the morning? I wouldn't. It's not a good course for them and it's sure not going to help the industry any.

Personally, I'd rather see someone step in and get Yahoo to start leveraging their many verticals. Yahoo has all the portal strength an engine could ask for, and yet they're not using it. Tapping into all of that is how Yahoo will succeed and grow. I still have faith in Yahoo and I don't want to see them sell out and become completely useless. Someone has to come along and challenge Google. Yahoo's sitting back at a pretty distant second right now, but Google's not going to reign forever. At some point, someone will come up and beat them. That will never be Yahoo if they hang up their gloves before the fight is over.

Hang in there, Yahoo. Don't be so quick to sell out. Even if Microsoft does keep sweet talking you back to the table.

Fun Finds

Copyblogger has launched a Twitter Writing Contest. You write your best 140 character story and the winner receives an iPod Nano 4 GB. Your story has to be exactly 140 characters. Over or under isn't going to cut it. Good luck!

Darren Rowse lists 12 Traits of Successful Bloggers even though the URL says eleven. Don't be fooled.

Posted by Lisa Barone on 05/19/08 at 4:41 PM | Comments (1)
See more entries in Branding, Microsoft, Yahoo