Accidental noindex: find pages blocked from Google

What noindex means and how to catch pages accidentally blocked from Google. We flag every noindexed URL in your audit.

3447

audited sites affected

What it means

A page flagged as noindex carries a directive telling search engines not to include it in the index. This can come from a meta tag in the HTML or an X-Robots-Tag header. The page is still fetched and rendered, but it won't appear in search results unless another signal overrides it.

Why it matters

The page wastes crawl budget because Googlebot still fetches and processes it every time. A noindex tag in the HTML cannot be overridden by JavaScript — it is read directly from the source. If the page is also blocked in robots.txt, the noindex is never even read, which can mask the real issue.

How to fix it

Open the page source and check for <meta name="robots" content="noindex"> in the <head>.
Verify the X-Robots-Tag header on the response — it may apply site-wide or to specific file types.
Confirm the page is not blocked by robots.txt — a disallowed URL never has its noindex read.
Remove the directive if the page should be indexed, or keep it if the page is intentionally excluded.

Example

Before — HTML / HTTP

<meta name="robots" content="noindex,follow">
<!-- or as a header -->
X-Robots-Tag: noindex

After — HTML / HTTP

<meta name="robots" content="index,follow">
<!-- remove the noindex if the page should rank -->

noindex removes the page from search. If it's a real page you want ranked, this is almost always an accidental staging/plugin setting.

When it's not a problem

This is fine when the page is a thank-you page, admin panel, or parameterized variant that genuinely should not appear in search. It is also a false alarm if the page is blocked in robots.txt — the noindex tag is redundant in that case.

How Crawlinx detects it

We read the meta robots tag and the X-Robots-Tag header. A noindex from either source means the page is asking to be excluded from search, so we report it with the directive we found.

How common is it?

3447 audited sites in our corpus currently show this issue. The breakdowns below show which platforms, gatekeepers, verticals and countries are most exposed.

FAQ

Why is my page noindexed?: Common causes: a leftover staging setting, a CMS/SEO-plugin default, or a broad X-Robots-Tag rule at the server/CDN. Check both the meta tag and the response header.

Related guides

Breakdowns

CMS

View breakdown →

Audit your own site — free

157 checks, internal PageRank, render-diff. No signup, results in ~30s.

Scan your site →