/ Learning Hub / HTTP Error Audit Guide

HTTP Error Audit: Find Every 4xx, 5xx and Redirect Issue

Broken URLs cost ranking, conversion and crawl budget. A 404 on a high-authority backlinked page is lost link equity. A 502 during Googlebot\'s crawl wave is a ranking signal regression. A redirect chain that goes A → B → C → D dilutes equity at every hop. The HTTP Error Audit crawls your entire site and surfaces every non-200 response, classified by type, with the referring pages and recommended fix. This guide covers each error class, why each matters, and how to fix them in priority order.

What the audit reports

4xx Client Errors404 Not Found, 403 Forbidden, 410 Gone, 401 Unauthorized.
5xx Server Errors500 Internal, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout.
SSL / TLS ErrorsExpired certs, hostname mismatch, broken chain, weak protocol.
Timeout ErrorsRequests that exceeded threshold (typically 30s).
Redirect IssuesLong chains, redirect loops, 302s that should be 301.
Referrer MapEvery internal page that links to a broken URL, plus external referrers if known.
Crawl WasteURLs that return errors but consume crawl budget repeatedly.
Recovery PriorityErrors sorted by impact — high-traffic, high-backlink, high-internal-link first.

404 Not Found — the most common

404 is the workhorse error. Caused by:

404 fix decision tree

For each 404 URL:
  Is there an equivalent new URL?
    Yes → 301 redirect to it
    No  → Is this URL still receiving real traffic?
              Yes (from backlinks) → 301 to closest topically-relevant URL
                                      OR build new content at that URL
              No                   → 410 Gone if intentionally removed
                                      404 if accidentally broken (fix the link)

NEVER:
  - 301 everything to the homepage (loses backlink equity routing)
  - Leave high-traffic 404s as 404 (lost equity)
  - Use 302 instead of 301 (signals temporary, not what you mean)

403 Forbidden — usually authentication or access control

403 returned to a crawler usually means: access control mistake (a public page got marked private), staging environment leaked into prod URL space, or WAF blocking legitimate crawl traffic. Fix at config level — usually NGINX/Apache directives or app-level auth middleware.

💡 If Googlebot is returning 403 on pages real users can access, your WAF is over-aggressive on bot traffic. Whitelist Googlebot, Bingbot and the AI crawler tokens (GPTBot, ClaudeBot, PerplexityBot) explicitly in your WAF rules.

410 Gone — the deliberate signal

410 says "this resource existed but is permanently removed; don\'t come back". Google de-indexes 410 URLs faster than 404 URLs. Use 410 for:

Don\'t use 410 for content that may return — use 404 (temporary unknown) or keep the page with an OutOfStock signal.

500 / 502 / 504 — server errors

These are infrastructure problems, not SEO problems — but they have SEO consequences. A 5xx during Googlebot\'s crawl wave is treated as "site temporarily unavailable" and ranking is temporarily reduced. Persistent 5xx leads to de-indexation.

CodeMeaningTypical cause
500Internal Server ErrorApp crashed, DB connection, unhandled exception
502Bad GatewayNGINX/proxy can\'t reach upstream app
503Service UnavailableMaintenance mode, rate limit hit, app overloaded
504Gateway TimeoutUpstream app took too long to respond

The audit reports 5xx errors with timestamp and frequency — patterns reveal whether the problem is intermittent (capacity), scheduled (maintenance windows), or systemic (specific URL or query patterns failing).

SSL / TLS errors

The audit checks certificate validity, expiry, hostname match, chain trust and protocol version. Common issues:

SSL errors are P0 — they block ALL traffic, not just SEO. Fix immediately via your certificate authority (Let\'s Encrypt, CloudFlare, paid CA).

Redirect chains and loops

Each redirect hop dilutes link equity and adds latency. The audit flags:

Fix: collapse every chain to a single redirect from origin to final destination. The audit\'s recommended fix lists the consolidation per chain.

Internal vs external broken links

The audit distinguishes:

Priority order

  1. SSL errors — P0, blocks all traffic
  2. 5xx errors — P0/P1, depends on volume and pattern
  3. High-traffic 4xx — pages that recently received clicks now returning 4xx
  4. High-backlink 4xx — pages with significant external links
  5. Long redirect chains — equity dilution and latency
  6. Crawl-waste 4xx — repeat-error URLs consuming budget
  7. Long-tail 4xx — low-traffic, low-link errors; fix when convenient

Frequently Asked Questions

How often should I run the HTTP Error Audit?
Monthly for stable sites. Weekly during active development or after releases. After any platform migration. The audit catches regressions early — a deploy that breaks 100 URLs is much cheaper to fix in week 1 than week 8 after Google has re-indexed the errors.
Should I redirect 404s to the homepage?
Almost never. Redirecting all 404s to the homepage loses backlink equity routing (Google treats the redirect as a soft 404), confuses users (they expected something specific), and creates a misleading signal that every URL is equally valuable. Redirect each 404 to its closest topically-relevant URL. If there\'s no relevant equivalent, leave it as a 404 (or 410 if permanently gone).
Is a 5xx error worse than a 4xx for SEO?
Temporarily worse, less permanently damaging. A 5xx tells Google \"site is broken right now, come back later\" and pauses ranking signals while it persists. A 4xx tells Google \"this specific URL is gone\" and removes that URL from the index. Persistent 5xx eventually causes de-indexation site-wide; persistent 4xx affects only the specific URLs. Fix 5xx fast because it scales; fix 4xx in priority order because it\'s contained.
Can I use the audit to find broken backlinks pointing to my site?
Yes. The audit reports external URLs that link to broken targets on your site — those are link-reclamation opportunities. Each external link to a 404 represents lost equity you could recover by 301-redirecting the broken URL to a current relevant URL. See the link reclamation workflow in the brand mention guide.

🔴 Audit every broken URL

Find every 4xx, 5xx, SSL and redirect issue with referrer maps and prioritised fixes.

Run HTTP Error Audit →
Related Guides: How to Fix HTTP Error Audit Findings  ·  Redirect Checker Guide  ·  Site Crawler Guide  ·  HTTP Error Audit Fixes
💬 Got a problem?