Page could not be fetched: why and how to fix it

What it means

The page failed to be retrieved by the crawler, typically because it returned a 5xx HTTP status, a network/DNS error, or a timeout during the fetch attempt.

Why it matters

Crawlers treat these as server errors rather than normal 4xx responses, so they slow down and delay re-crawling. If the problem persists, the URL is eventually dropped from the index. On robots.txt itself, a 5xx halts crawling of the entire site for roughly 12 hours, which is worse than a 404.

How to fix it

1. Check the HTTP status code returned by the page—if it's 5xx, fix the server error or add a Retry-After header for maintenance windows.
2. Verify DNS resolution and network connectivity from Google's perspective using the URL Inspection tool.
3. Ensure the page is not blocked by robots.txt, since a 5xx on robots.txt stops crawling the whole site.
4. Return 404 or 410 for genuinely gone pages, and avoid redirecting all missing pages to the homepage, which becomes a soft 404.

When it's not a problem

A single transient fetch failure is not a ranking penalty—Google only drops the URL if the error persists across multiple crawl attempts. Small sites crawled on the same day rarely need active crawl budget management.

How common is it?

Not yet observed in our audit sample. We publish this explainer because the check runs on every crawl; once an audited site trips it, this page will break the issue down by platform, gatekeeper, vertical and country.