Fixing Blocked or Noindexed Hreflang Targets

Why a blocked target breaks more than one page

Hreflang is confirmed in pairs. When the British page lists the German page, Google only trusts that link once it has crawled the German page and found a line pointing back. A blocked target interferes with that process in two separate ways, and it helps to keep them apart:

• The target cannot be shown. A page with noindex is dropped from the index, so there is nothing for Google to swap in for German searchers, whatever the annotations say. Google has said it ignores hreflang pointing at pages it will not index.
• The target cannot vouch for the others. A page disallowed in robots.txt is never fetched, so Google never reads its head. The return tags it contains, which the rest of the cluster relies on, are invisible. The British page's link to Germany is left unconfirmed.

The second effect is the one that catches people out. A robots.txt block on /de/ looks like a German problem, but it quietly removes confirmation for every link that runs to or from the German pages, across every other locale.

The ways a target can be blocked

The Hreflang Checker reports a target as blocked when it cannot be used by Google. These are the causes you will meet, roughly in order of how often they turn up:

Pages that return 404 or redirect elsewhere are a related but separate problem, handled in the guide to hreflang pointing at 404s or redirects. This guide is about pages that exist and work for people but are shut off to search engines.

Diagnosing each block precisely

Start from the checker's list of blocked targets, then confirm the exact mechanism for each one before you change anything. The same symptom can have three different causes, and fixing the wrong one wastes a release.

robots.txt

Open https://example.com/robots.txt and read every group that applies to Googlebot. Remember that Google follows the most specific matching group for its user agent, and within that group the longest matching rule wins. A group for Googlebot replaces the * group entirely for Google; it does not add to it. Search Console's robots.txt report shows the version Google last fetched, which may be older than the file you are looking at.

noindex in the HTML or the header

View Source misses headers, so check both at once from the command line:

curl -sI -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://example.com/de/preise/ | grep -i "^HTTP\|x-robots-tag"
curl -s https://example.com/de/preise/ | grep -io '<meta[^>]*robots[^>]*>'

Look for noindex in either place, and also for none, which means noindex and nofollow together. A robots meta tag aimed at googlebot specifically counts as well.

Authentication and firewalls

A 401 means the server wants credentials; a 403 usually means a firewall, bot-protection service or IP allow list refused the request. Run URL Inspection's live test in Search Console on the target: if Google's own fetch is refused, the problem is real, whatever your browser shows while you are logged in.

Staging addresses

Search the live HTML and sitemaps for hosts that should never appear in production, such as staging., dev., preview. or a hosting provider's temporary domain. One stray host in a template is enough to put a locked address into every cluster on the site.

Should this locale be indexable at all?

Every blocked target falls into one of two groups, and the fix is opposite for each. Decide deliberately rather than removing blocks by reflex.

The rule behind the table is simple: a page should be in a hreflang cluster only if you want it to be found in search. If you want it hidden, it should not be named by any other page.

Common patterns and how to repair them

A robots.txt rule that catches a locale by accident

Robots rules match from the start of the path, so a short prefix matches more than you expect. A rule written to hide an internal search page can also hide a whole language:

# Meant to block /search, but it matches every path
# starting with /s, including /sv-se/ and /shop/
User-agent: *
Disallow: /s

# Specific and safe
User-agent: *
Disallow: /search
Disallow: /*?q=

The same happens with wildcards. Disallow: /*? blocks every address with a query string, which is harmless until a site uses ?lang=fr for its locales. If your locales are parameter-based, either move them to paths or allow the parameter explicitly with a longer, more specific Allow rule.

A blanket noindex on a new language

Translation plugins and headless CMS setups often let editors mark a whole language as hidden while it is being prepared, and that setting adds noindex to every page in it. The annotations, meanwhile, are generated from the list of configured languages, so the hidden language is advertised everywhere. Either publish the language properly or change the generator so that it only lists languages marked as public.

A header left over from staging

An X-Robots-Tag: noindex header is a sensible way to keep a staging site out of search, but server configuration is often copied between environments. If the header rule is tied to a directory or a location block rather than to the staging hostname, it can travel to production. Tie it to the host:

# nginx: noindex only on the staging host
server {
    server_name staging.example.com;
    add_header X-Robots-Tag "noindex, nofollow" always;
}

A country gate that stops crawlers

Some shops show visitors from outside a country a "not available in your region" page, or send them to a sign-in page. Googlebot mostly crawls from United States addresses, so a gated Irish or Australian locale may be refused to Google every time. Let crawlers and visitors read the content, and use a banner rather than a hard block to explain delivery limits.

Platform notes

• WordPress. WPML and Polylang can hide languages from visitors or search engines while they are being translated, and an SEO plugin may add noindex to specific post types or archives in one language only. Check the language settings and the SEO plugin's per-language defaults together. The WordPress hreflang guide covers plugin clashes in more depth.
• Shopify. A market can be set up with a domain or subfolder before it is ready. Shopify's own robots.txt is generated from a theme template, so an app or a customised robots.txt.liquid may be the source of a disallow rule.
• Next.js and Nuxt. Environment variables often decide whether a noindex tag is added. A build that reads the wrong variable can ship noindex to production for one locale domain only. See hreflang in Next.js and Nuxt.
• CDNs and firewalls. Cloudflare, Sucuri and similar services can challenge crawlers by country or by rate. A 403 or a challenge page served to Googlebot on one locale host is a block, even though it never appears in your code.

The fix, step by step

Keeping locales open, and keeping drafts out

• One list of public locales. Generate annotations, sitemaps and the language switcher from a single configuration that marks each locale as public or not. A locale that is not public is never named.
• Release checks. Add a check to your deployment that fetches one page per locale and fails if it sees noindex in the head or header, a 401 or 403, or a staging hostname in any hreflang link.
• Review robots.txt changes. Treat robots.txt like code: keep it in version control and have someone read every change for prefixes that are shorter than they need to be.
• Soft launches without hreflang. If you want to share a new locale privately before launch, protect it with a password and leave it out of the annotations. Add it to the cluster on the day it becomes public, not before.

For the underlying mechanics of disallow rules and noindex outside hreflang, see fixing noindex and disallow conflicts.