XML Sitemaps in SEO – Part 1
Many people consider XML Sitemaps as an optional extra, but the fact is XML Sitemaps help not only in the indexation of your pages by providing a full list of URLs on your website, but also to provide other useful information to search engines about your site. For example, by submitting “/” as your root domain instead of “default.aspx” through the XML Sitemap, search engines like Google might just figure out “/” is the main URL and this may help with the canonical issues.
Other benefits of XML Sitemaps include:
- Help improve the crawl rate and indexation on a site
- Help the crawling of newer content
- Help spot problems or raise trouble flags (In the case of Google, problems are reported in Webmaster Tools)
In XML Sitemaps, we generally include all the URLs of our site to help search engines discover most of our URLs. We can also create XML Sitemaps for other content like videos and mobile web pages to make them easily discoverable by search engines. In this post, I will cover most of the XML Sitemaps and will talk about videos and mobile sitemaps in a future post.
Location of XML Sitemap
By default, the location of XML Sitemap can be assumed as your-site/sitemap.xml. If we have multiple sitemaps or our CMS generates files with some unique names, all we have to do is mention it by adding the following line in the robots.txt file:
Always remember, Google is not the only search engine out there. For Google and Bing, XML Sitemaps can be easily submitted via Google Webmaster Tools and Bing Webmaster Tools but for all other search engines, the only way to figure out the location of your XML Sitemap is through the robots.txt file.
When is XML Sitemap helpful?
XML Sitemaps are only there to help search engines discover URLs for your website. It is not guaranteed that all the URLs submitted using an XML Sitemap will be crawled and indexed. For example, if we have a 500 page site and search engine spiders can only crawl the first 200 of the pages, it is not a good idea to force the other 300 pages via a sitemap. The solution would rather be fixing the crawlability problems like making sure every page has an appropriate number of links and avoiding deep links. Also, if there is an indexation problem, we should focus on the other factors like the quality of the content, link building strategies and social interactions.
Creating a XML Sitemap
It is highly recommended to use Sitemap 0.9 protocol as defined by sitemaps.org. All the major search engines accept this standard protocol. There are many free tools available that can be used to create an XML Sitemap. Xenu is one tool that crawls websites and also can be used to create an XML Sitemap.
Optimising XML sitemap
Sometimes, we might end up having thousands of URLs in a sitemap file. Google recommends not having more than 50,000 URLs or keeping the file size less than 10 MB. If our sitemap exceeds this limit, it is important to split the large file into multiple files and create a sitemap index file that lists all the sitemap files on your site. We will only need to submit the sitemap index file to the search engines. To optimise the sitemap files, we can also compress the file using gzip.
It is important to consider using an XML Sitemap especially for huge sites to make sure all the URLs are discoverable by search engines. There are results we have seen where a small change nearly tripled the indexation percentage. Stay tuned and keep reading our blogs for more SEO.