How to check if a page is no index?
Where noindex directives can hide
Noindex directives can live in two places, and checking only one while missing the other is a common mistake. The first is the HTML <head> section: a meta tag like <meta name='robots' content='noindex'> or similar tells crawlers not to index the page. The second is the HTTP response header: a server can send an X-Robots-Tag header with a noindex value, which has the same effect but isn't visible in the page's HTML source at all — it requires inspecting the HTTP headers, which means using developer tools or a dedicated checker. Both are equally respected by Google.
How noindex tags end up on pages accidentally
The most common cause of accidental noindex tags is CMS or theme settings. Many content management systems include a checkbox or setting to 'noindex' a page, and these are sometimes applied by default to category pages, tag pages, paginated pages, or search result pages. When a developer migrates a site or switches themes, they sometimes forget to transfer these settings correctly, accidentally leaving noindex on pages that should be indexable. A full site crawl that reports the noindex status of every URL is the most reliable way to audit for accidental noindex at scale.
Should you ever intentionally noindex pages?
Yes — noindex is a legitimate and useful directive for pages that genuinely shouldn't appear in search results. Common intentional uses: internal search result pages (e.g. site.com/search?q=...), paginated versions of long archives, thank-you and confirmation pages, login or account management pages, and any pages that contain content you don't want publicly searchable. The key discipline is being intentional about which pages carry noindex directives, auditing for them regularly to confirm they're still correctly applied, and ensuring no pages are accidentally noindexed that should be visible in search.
Key Takeaways
- Noindex can appear in HTML meta tags OR in HTTP response headers — check both locations.
- Accidental noindex from CMS settings is common after redesigns or theme switches.
- Intentional noindex is valid for search pages, confirmation pages, and account areas.