Get Free Quote
« Rob Kerry of Ayami... | Blog home | Cindy Krum,... »
April 4, 2009

Priyank Garg on Bot Herding

  • Print Friendly

Next up on the SMX Sydney Technical SEO Track is "Bot Herding" by Priyank Garg, Director, Product Management at Yahoo! Web Search

Priyank Garg is Director of Product Management at Yahoo! India. He is leading Product Management efforts for the Yahoo! Search team based out of Bangalore. [Aiden, our local Yahoo search universe master, made Garg’s visit happen for SMX this year so a big thanks for him for that effort. – Kate] Priyank has published multiple papers and filed for more than 12 patents over his professional career.

Gard explains that Robots.txt is the baseline must do for “bot herding” (telling search engines what to do when visiting a site. Robots.txt allows publishers to tell search engine crawlers which content should be included in search results. The independent site answers a lot of the most frequently asked robots.txt related questions

Priyank suggests that all webmasters sign up for Yahoo Site Explorer, it’s a useful tool to let you privately see how Yahoo is indexing your content.

More information at and an explanation of the Yahoo Dynamic URL Rewriting content tool

  • Print Friendly

Comments are closed.

Get Started
Learn SEO

Subscribe to Blog Updates

Get email notifications when we publish new blog posts, usually two to three times a week.


We respect your privacy and never share your email address

Free Executives Guide To SEO
By continuing to use the site, you agree to the use of cookies. AcceptDo Not Accept