What Is an XML Sitemap and How Do I Make One?
An XML sitemap is a file that webmasters create and put on their site to tell search engines like Google and Bing about the pages, images and videos that are on the site. This content list works like a map, helping ensure more thorough crawling and indexing.
The XML sitemap is created using XML (Extensible Markup Language), a type of markup language used on the web in which tags share information.
Not only do XML sitemaps tell the search engines all the URLs that you want indexed and crawled, but they also provide other information such as how frequently you update the pages.
An XML sitemap differs from an HTML sitemap in that the XML sitemap is just for search engines. On the other hand, an HTML sitemap is a webpage on your site that contains links to help visitors navigate to the important pages on your site.
Now that you have definitions in hand, let’s talk about:
- Why XML sitemaps are an SEO best practice
- Special types of sitemaps to consider
- How to create an XML sitemap
- How to submit an XML sitemap
- A case study and final thoughts
XML Sitemaps: An SEO Best Practice with Benefits
For SEO purposes, you must build an XML sitemap and keep it up to date to help ensure that search engines index and crawl all the important pages on your site.
While some view an XML sitemap as a “nice to have,” it’s actually an SEO best practice for every site to have at least one — even though Google says that sites under 500 pages might not need it.
Sure, the search engines should be able to find all the pages by following links on the site, but the reality is that many sites don’t follow proper linking architecture. So it can be hard for search engines to discover the content.
Benefits of XML sitemaps include:
- They improve the crawl rate and indexation on a site.
- They can help you spot problems (for Google, check Search Console).
- They provide other useful information to search engines about your site.
- They alert search engines to new pages and hopefully get them indexed sooner than if you waited for search engines to find them.
Now that we’ve gone over some benefits, it’s worth noting what XML sitemaps cannot do:
- An XML sitemap does not guarantee that a search engine will index or crawl all pages, nor will it pass any link popularity or help with subject theming.
- An XML sitemap will not impact your rankings. But having a higher number of pages indexed in the search engines may increase your chances of ranking.
- An XML sitemap should not be relied upon as a way to fix crawling issues. If there is an indexation problem, also look at other factors like the architecture of the site or the quality of the content and its links rather than relying on an XML sitemap alone.
Special XML Sitemaps to Know
Other than the standard XML sitemap, it’s good to know about key sitemaps specific to content such as news, images and video. Here, we’ll primarily go over Google sitemaps. You can learn more about Bing sitemaps here.
News XML Sitemaps
If you are a publisher of news-related content and don’t have a news sitemap, you may not be getting the visibility you want. For articles that have been published in the past two days, a news sitemap contains URLs for them. Create news sitemaps in addition to your generic XML sitemaps. For more, learn how to create a news sitemap.
Video XML Sitemaps
Video sitemaps can help Google find and understand your video content by telling it exactly where and what the video content is on your site. Video content includes webpages that embed videos, URLs to video players, or the URLs of raw video content. If Google cannot discover the video content at the URLs provided, it will ignore them. Note that while Google recommends using video sitemaps and schema.org’s VideoObject to mark up videos, it also supports mRSS. You can also view Bing’s guidelines on video sitemaps here.
Image XML Sitemaps
How to Create an XML Sitemap
You could create a sitemap manually, but using a sitemap generator makes the job easier. And to help you there are many good third-party tools for creating XML sitemaps. One is Microsoft Bing’s free server-side Bing XML Sitemap Plugin, which can automatically generate two types of XML sitemaps that any search engine can read:
- Comprehensive sitemap, which includes all files (except any you disallow in your robots.txt file)
- Recently updated sitemap, which includes URLs of changed files only (useful for your own tracking or for prioritizing the pages that search engines should crawl)
Here’s a useful video from Google on creating an XML sitemap:
We also cover how to create sitemaps (both XML and HTML versions) in our SEO Guide.
For Large Websites
XML sitemaps are especially useful for large sites to make sure all the URLs are discoverable by search engines.
Large websites may need to break their list of URLs into many XML sitemaps. This ensures that the number of page URLs per sitemap doesn’t exceed the limit.
XML sitemaps can contain up to 50,000 page URLs.
You can have separate XML files by media type if you have original videos, news, images, etc. that you want indexed. So for example, if you have videos on your site, create a specialized video XML sitemap to help make sure the search engines find your video files.
As a bonus, if you break down your XML sitemaps into smaller sitemap files, maybe by site sections, it allows you to watch your indexation performance for each section of your site and identify where indexation issues exist.
You can then create a sitemap index file that lists all the sitemap files on your site. To optimize sitemap files, you can also compress the file using gzip.
XML Sitemap Tips
The required XML tags are: <urlset>, <url>, and <loc>. The tags <urlset> and <url> are for formatting the XML, and <loc> is for identifying the URL.
Optional meta data tags are:
- <lastmod> – last modified date
- <changefreq> – how often the page changes (such as hourly, daily, monthly, never)
- <priority> – how important the page is from 0 (the lowest) to 1 (the highest)
Site owners aren’t required to use the optional tags, but the engines may consult them when deciding how often they should recrawl pages. Google states that it does not use the <priority> or <changefreq> tags at all. While Google may consider <lastmod>, it does not base decisions on this tag.
If you use these tags, keep them accurate to help the search engines better crawl your site. Pages that you are optimizing should be set to a higher priority. If you have archived pages that you haven’t updated in years, set to a low priority with a <changefreq> of “never.”
Upload to the Site
Once you have created the sitemap file, upload it to the root of your website (for example: https://www.your-domain-name.com/sitemap.xml). Now it’s time to let the search engines know about it using your robots.txt file.
A robots.txt file is simply a text file saved at the root of your website that gives instructions to visiting search engine spiders.
Your robots.txt file should look like this, with a sitemap directive line for each of your different XML sitemaps:
If you have multiple sitemaps, or if your CMS generates files with some unique names, then all you have to do is mention them by name in the robots.txt file, one per line. Or if you have created a sitemap index file, then you can specify just the index file location in the robots.txt and list all your separate sitemap files in the sitemap index.
Submitting a Sitemap
Some people prefer to submit the sitemaps manually. This is primarily due to timing. A submission is known to start the indexing instead of waiting for the search engine to give you a turn.
Another reason to submit a new sitemap to Google is to check it for errors. Google tries to continue parsing a sitemap file even if it has minor errors. However, if the XML is badly formed, then it could cause Google to ignore all entries after the badly formed entry (like a missing “>” or “</url>” tag). The Sitemaps report in Search Console will tell you if any problems were encountered, such as:
Has errors: The sitemap could be parsed but has one or more errors; any URLs that could be parsed from the sitemap will be queued for crawling.
You can proactively submit your XML sitemap(s) to Google and Bing as follows:
- Google: Log in to your Google Search Console account and go to Sitemaps.
- Bing: Log in to Bing Webmaster Tools. Then see the Sitemap widget on the dashboard or go to the Sitemaps feature.
XML Sitemap Case Study
After diagnosing that a client with a large website had only 20% of pages indexed, we implemented several tactics to help. We resubmitted their standard XML sitemap and fixed a large number of errors coming up on the client’s Search Console account.
We also submitted specialized XML sitemaps and implemented canonical tags throughout the entire site, as it had a large amount of duplicate content.
Indexation results jumped from 24% to 68%! And this percentage keeps growing, resulting in significant improvements in organic search traffic.
The goal of XML sitemaps is to help search engines crawl efficiently and thoroughly. You facilitate this by creating a sitemap and using the appropriate tags so the engines can understand how to best crawl your site.
As a final note: Be sure to keep your XML sitemaps up to date. If you add or remove pages, make sure your sitemap reflects that. You should also check Google Search Console frequently to ensure that Google is not finding any errors in your sitemap.
You can find more information about the sitemaps protocol at sitemaps.org.
If you need help with your website’s organic search performance, contact us for a free quote for SEO services.