The noindex challenge
Matt performed a 100 percent in-no-way-scientific test to see if the engines actually respect those little “noindex” commands you’ve been confidently stapling across your super secret Web stuff in order to keep it out of the index. Turns out, not all the engines pay attention to the command. Here’s what Matt found.
- Google doesn’t show the page in any way
- Ask doesn’t show the page in any way
- MSN shows a url reference and Cached link, but no snippet. Clicking the cached link doesn’t return anything.
- Yahoo! shows a url reference and Cached link, but no snippet. Clicking on the cached link returns the cached page.
Yahoo!, you sneaks! I knew you were looking at me funny during SES. It’s nice to know that at least Ask (go Ask!) and Google keep their word (and your secrets).
Interestingly, over at SEW Danny Sullivan expands on Matt’s comments and points out that Google treats robots.txt and noindex differently, even though they’re technically supposed to do the same thing. It’s important to know, because Danny says, if you use a robots.txt file instead of the nofollow, Google does show the page in some ways. Hmm, I wonder why the discrepancy? It’s definitely something webmasters should be aware of.
Check out Matt’s post to read his full “study”.