How to Fix Blocked Hreflang Targets

Hreflang declares "this URL is an indexable locale variant". If the target is blocked by robots.txt, blocked by noindex meta, or behind auth, Google can't honour the annotation. The cluster signal weakens or fails entirely. Two outcomes apply per finding: either the page should be indexable (remove the block) or it shouldn't be in the cluster (remove from hreflang config). Both fixes work; the wrong fix is leaving both signals contradictory.

1. The block types

2. Audit

# For each hreflang URL, check if robots.txt allows it
URL="https://example.com/fr/about"
ROBOTS_URL="https://example.com/robots.txt"

python3 -c "
from urllib.robotparser import RobotFileParser
rp = RobotFileParser()
rp.set_url('$ROBOTS_URL')
rp.read()
print('Crawlable:' if rp.can_fetch('Googlebot', '$URL') else 'BLOCKED:', '$URL')
"

# Check meta robots and X-Robots-Tag header
curl -sI https://example.com/fr/about | grep -i "x-robots-tag"
curl -s https://example.com/fr/about | grep -oE '<meta[^>]*robots[^>]*>'

# Both should be absent or set to "index, follow"
# If "noindex" appears, target is blocked

3. Decide intent

Per blocked target, ask: should this URL be indexed by Google?

YES, it should be indexable

1. Remove the block (robots.txt rule, noindex meta, auth, etc.)
2. Keep hreflang as-is
3. Verify by requesting re-crawl in Search Console

NO, it shouldn't be indexed (staging, draft, low-quality translation, internal-only)

1. Keep the block in place
2. Remove the variant from hreflang cluster config
3. Redeploy — sibling pages no longer reference the blocked variant

4. Common fix patterns

Pattern: staging URLs in production hreflang

// BAD: production page references staging
'fr': 'https://staging.example.com/fr/about'  // robots-blocked

// FIX: update cluster to production URL
'fr': 'https://example.com/fr/about'

Pattern: overly broad robots.txt

# BAD: robots.txt blocks legitimate hreflang targets
User-agent: *
Disallow: /fr/internal/   # OK
Disallow: /fr/            # TOO BROAD — catches everything

# FIX: tighten the rule
User-agent: *
Disallow: /fr/internal/
# Public /fr/* now crawlable

Pattern: blanket noindex on a locale

<!-- Some sites add noindex to all pages in a soft-launch locale -->
<meta name="robots" content="noindex">

<!-- If launched, remove the noindex -->
<!-- Until launched, keep noindex AND remove variant from hreflang config -->

5. Soft-launch pattern (recommended)

When launching a new locale, the right sequence:

1. Build locale content in production with noindex meta
2. QA, translation review, link checking
3. Remove noindex meta on the new locale
4. Add new locale to hreflang cluster config
5. Redeploy all variants — cluster now includes the new locale
6. Submit to Search Console for re-crawl

This avoids the window where blocked variants are advertised. Hreflang only references indexable URLs.

6. Per-platform reminders

WordPress: WPML draft handling

// WPML offers "complete translation" workflow.
// Posts marked draft/incomplete shouldn't appear in hreflang.
// Confirm WPML setting: "Include only complete translations in hreflang"

Next.js i18n: build flags

// Conditionally include locales based on env
const locales = process.env.NODE_ENV === 'production'
  ? ['en', 'fr', 'de']
  : ['en', 'fr', 'de', 'es-beta'];

module.exports = {
  i18n: { locales }
};
// Beta locale doesn't appear in production hreflang

7. Verify resolution