Sitemap “Couldn’t fetch” and the other sitemap errors

The Sitemaps report in Search Console lists every sitemap you submitted, with a status for each. There are three, and they answer different questions.

Click a sitemap in the report to open its details. One that could not be fetched shows Sitemap could not be read. One that was fetched but has problems shows Sitemap can be read, but has errors, with the errors listed under it.

Google does not keep trying forever. Its help page says that when a sitemap fetch fails, Google retries for a few days and then stops. A fix you make after that is read when you submit the sitemap again, so resubmitting is part of the fix.

It lists nothing

A generator that runs before its data is ready writes a valid <urlset> with no URLs in it. A build step that ran before the content was available will do it, and so will a database that was unreachable for one deploy. The file parses and answers 200. Search Console reports Empty sitemap the next time Google reads it, which may be days later.

In Next.js, for example, app/sitemap.ts returns the list of URLs. If the function catches a failed request to your API and returns an empty array, the result is still valid XML with nothing in it.

It is cut off partway

A generator that times out while writing leaves half a file. The response ends in the middle of a tag, no parser accepts it, and Google reports Parsing error. The end of the file tells you at once: it should finish with </urlset>, or </sitemapindex> for an index.

curl -s https://www.example.com/sitemap.xml | tail -c 100

It lists far fewer URLs than yesterday

Search Console shows no error for this one. The file is valid, so the status stays Success. Yesterday it listed 4,231 URLs and today it lists 1,407. That could be a deliberate clean-up or a generator that failed halfway, and from outside the two look the same. A count kept over time is the only way to see it.

One sitemap behind the index stopped answering

Large sites split their sitemap into parts and list the parts in an index. The index can stay valid while one of the sitemaps it names returns 404, and a check that reads only the index sees nothing wrong. Step 6 fetches each one:

The gzip file is not what its name says

A file called sitemap.xml.gz should arrive as gzip bytes. A server can compress it a second time on the way out, or serve plain XML under the .gz name. Decompress it once and parse the result. If that fails, Google is probably getting the same thing you are.

curl -s https://www.example.com/sitemap.xml.gz | gunzip -c | xmllint --noout -

A robots.txt that names two sitemaps. The lines can go anywhere in the file:

User-agent: *
Disallow: /cart

Sitemap: https://www.example.com/sitemap_index.xml
Sitemap: https://www.example.com/sitemap-images.xml

Google calls submitting a sitemap “merely a hint”. It tells Google which URLs exist and when they changed. Google still decides what to crawl, when, and whether to index it.

A sitemap is a list of URLs for crawlers. It does not make a page rank, and losing it does not remove pages Google has already indexed. Google also finds pages by following links, so most of an established site keeps being crawled without one.

What a broken sitemap costs is time. New pages, pages deep in the site and pages with few internal links are the ones a crawler is least likely to reach by links, and those are the pages the sitemap was helping. On a site that publishes every day, a sitemap that has been broken for a month means a month of new pages that took longer to appear in search.

The same reasoning applies to lastmod on a sitemap that works. Google uses the date when it is consistently accurate. A generator that stamps every URL with today's date on every run gives Google a value it has no reason to trust.

1. 01

   Run the checks again

   Step 1 should print 200, an XML content type, 0 redirects and the URL you submitted. Step 3 should print nothing.

2. 02

   Submit the sitemap again

   In the Sitemaps report in Search Console, submit the same URL again. Doing that for a sitemap Google already knows does no harm.

3. 03

   Watch the status

   Google gives no time frame for reading a resubmitted sitemap, so check back over the next few days. For Temporary error, Google's advice is to submit again if it is still there after several hours.

4. 04

   Update robots.txt if the URL changed

   If the sitemap moved, change the Sitemap: line in robots.txt, submit the new URL, and remove the old one from the report so it stops showing an error.

Why does my sitemap open in a browser when Search Console says Couldn't fetch?

Your browser is not a crawler. It sends cookies and passes bot protection, and a crawler gets neither. Test with the step 1 command, which sends a plain request. If that also gets 200 with XML, the error may be one of the temporary ones Google mentions: submit the sitemap again and check over the next few days.

How long does Google take to read a sitemap?

Google does not give a time. Its help page says a failed fetch is retried for a few days and then stopped, and that a temporary error is worth resubmitting if it is still there after several hours. After you fix a sitemap, submit it again rather than waiting for Google to come back.

Does the sitemap have to be called sitemap.xml?

No. Any URL works when you submit it in Search Console or name it in robots.txt with a Sitemap: line. The name only matters to tools that have to guess, and they usually try /sitemap.xml first.

Can a site have more than one sitemap?

Yes. Submit each one, list them all in robots.txt (there is no limit on Sitemap: lines), or list them in a sitemap index of up to 50,000 sitemaps. An index cannot list another index.

Should every page be in the sitemap?

List the URLs you want in Google, each at its final, canonical address. Google reports URLs that redirect too many times as URLs not followed, and URLs blocked by robots.txt as an error of their own. A noindexed URL in a sitemap gives Google two opposite instructions, so leave those out too.

Do I need to resubmit the sitemap every time I add a page?

No. Keep the file current and the lastmod dates accurate. Submit it again after fixing an error, or when its URL changes.

Is a sitemap error the reason my pages are not indexed?

Not usually by itself, because Google finds most pages through links. If pages are missing from Google, the page indexing report gives a reason for each one, and the guide to those reasons explains them. A sitemap error is not one of them.