What is an XML sitemap, and what should yours contain?

A search engine discovers the pages on your site by following links, much as a person browsing at leisure would. That works well on a small, well-linked site, and less well everywhere else: a shop with thousands of products, a blog with years of archive, or a new page nobody links to yet.

The XML sitemap is the list you hand to the search engine so it does not rely on that exploration alone: “these are the URLs I want you to know about, and this is when each one last changed”.

What it looks like

It is an XML file with a very simple structure, defined in the public sitemaps.org protocol shared by Google, Bing and the other search engines:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://yourdomain.com/services/</loc>
    <lastmod>2026-03-14</lastmod>
  </url>
  <url>
    <loc>https://yourdomain.com/blog/choosing-a-host/</loc>
    <lastmod>2025-11-02</lastmod>
  </url>
</urlset>

Every <url> must have its <loc>, the full address with https:// and the domain. Relative URLs do not count. And because it is strict XML, one badly escaped character (the classic & in a URL with parameters, which must be written &) invalidates the whole file.

The usual practice is to publish it at /sitemap.xml and declare it in robots.txt with a Sitemap: line, as well as submitting it in Search Console and Bing Webmaster Tools. That way any crawler finds it without guessing.

Does it guarantee indexing?

No, and expectations are worth adjusting. A sitemap is a suggestion: it helps the search engine discover URLs and decide when to revisit them, but it does not force any of them into the index. Google still assesses each page on its own merits, and a thin, duplicated or poorly linked page can stay out even if it is in the sitemap.

That said, for a site whose content changes or grows, a well-maintained sitemap is one of the technical improvements with the best ratio of effort to result.

When you need a sitemap index

The protocol sets two limits per file: 50,000 URLs and 50 MB uncompressed. You can serve it gzip-compressed to save bandwidth, but the limit applies to the uncompressed size.

When a site goes past those figures, or simply when it helps to organise things, you use a sitemap index: a file that, instead of page URLs, lists other sitemaps.

<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://yourdomain.com/sitemap-products.xml</loc>
    <lastmod>2026-04-30</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://yourdomain.com/sitemap-blog.xml</loc>
  </sitemap>
</sitemapindex>

Splitting by content type has a practical benefit even if you hit no limit: in Search Console you see separately how many URLs from each sitemap are indexed, which tells you straight away whether the problem lies with products, the blog or the categories.

The fields that actually matter

The protocol defines four tags per URL, but they do not carry equal weight:

Tag	What it was designed for	What it does today
<loc>	The page address	Mandatory
<lastmod>	Date of last modification	Used by Google and Bing to decide what to recrawl, if it is reliable
<changefreq>	How often it changes	Ignored by Google
<priority>	Relative importance (0.0 to 1.0)	Ignored by Google

lastmod is the one that deserves care. It should reflect when the page content genuinely changed, in ISO 8601 format (2026-03-14, or with time and time zone). And here is the trap: many generators stamp today’s date on every URL each time the sitemap is rebuilt. If the search engine notices your dates do not match real changes, it stops trusting them, and you lose precisely the signal that made the file useful. The same happens with dates in the future.

Changing the footer or the copyright year is not a content change. Correcting a price, adding a section or rewriting a paragraph is.

URLs that should stay out

The underlying rule is simple: a sitemap should list only canonical URLs that return 200 and that you want to see in Google. Anything else sends the search engine mixed signals.

So leave out:

• URLs that redirect. If /summer-deal redirects to /offers/, the second one goes in the sitemap.
• Pages returning 404 or 410. The most common case after deleting products or posts without regenerating the file.
• Pages with noindex. You are asking for them to be indexed and not indexed at once.
• URLs blocked in robots.txt. The search engine cannot read them, so it cannot assess them either. This is covered in robots.txt and AI crawlers.
• Non-canonical versions. Variants with tracking parameters, shop filters, http:// on a site served over https://, or a mix of with and without www.
• URLs on another domain. A sitemap can only declare addresses on its own host; the rest are discarded.

A sitemap full of surplus URLs does not penalise you as such, but it makes the coverage reports harder to read and dilutes the file’s value as a signal.

Keeping it current

A hand-written sitemap goes stale within weeks. The sensible approach is for the site itself to generate it from its published content: almost every CMS and framework does this out of the box or with a plugin. The question is not whether it exists but whether it reflects reality:

• Do URLs disappear when a page is deleted or unpublished?
• Does lastmod come from the real edit date or from the build date?
• Do the URLs match the canonicals each page declares?

After a migration it is especially important to review it, because the new sitemap often keeps old routes, or the old one stays published. It is one of the standing checks in a well-planned migration.

Checking yours

The sitemap checker looks for a domain’s sitemap (at the usual paths and in robots.txt), walks the index if there is one, counts the URLs and reviews what this article describes: valid XML, absolute addresses on the same domain, lastmod dates that are missing, malformed or in the future, duplicates, and tags Google no longer uses. It also requests a sample of the URLs to see which ones really respond and which redirect or fail.

If your sitemap is declared in robots.txt, the robots.txt tester confirms that line is written correctly and that you are not blocking the very URLs you are asking to have indexed.

---

A good sitemap is boring: only live, canonical URLs, with dates that tell the truth, regenerated automatically whenever something changes. If yours looks like that, the search engine will treat it as a reliable source rather than noise.