How to Find Every 400 & 500 Error on Your Website

What are 400 and 500 HTTP errors?

Every time a browser or search engine requests a page from your server, the server responds with a three-digit status code. Codes starting with 4 are client errors — the request was wrong or the page does not exist. Codes starting with 5 are server errors — something broke on your end.

Common 4xx errors

Common 5xx errors

Do 400 and 500 errors hurt your SEO?

Yes — and in multiple ways.

Crawl budget waste

Google allocates a crawl budget to every site — a limit on how many pages it will crawl in a given period. Every 404 Googlebot visits is a wasted crawl. On large sites with hundreds of broken URLs, this means important pages get crawled less often or not at all.

Lost rankings

If a page that previously ranked starts returning 404, Google will eventually remove it from the index. Any backlinks pointing to that URL stop passing authority. If you do not redirect the URL, those rankings and that link equity are gone.

Poor user experience

Users who land on a 404 page leave immediately. High bounce rates on error pages are a negative user experience signal. On e-commerce sites, a 404 on a product page means a lost sale.

How do you find all HTTP errors on your site?

There are three main methods — ranked by thoroughness:

1. HTTP Error Audit tool (fastest and most complete)

The HTTP Error Audit at aiwebpageseo crawls your entire site, follows every internal link, and records the HTTP status code returned by each page. Results are shown in a filterable table — filter by 4xx errors, 5xx errors, redirects, or 200 OK. You can export to CSV for a full report.

It works by starting at your homepage, extracting all internal links, visiting each one, and repeating — exactly as Googlebot does. The scan size is tiered by number of pages:

2. Google Search Console

Google Search Console's Coverage report shows pages returning 404 or other errors that Googlebot has encountered. The limitation: it only shows pages Google has already tried to crawl. New broken pages may take days or weeks to appear. It also does not show errors on pages that are not yet in Google's index.

3. Server error logs

Your web server logs every request and its status code. Parsing your NGINX or Apache access logs will reveal every 4xx and 5xx response across all traffic — not just Googlebot. The downside is that logs require technical access and manual processing, and they do not show you which pages linked to the broken URLs.

How do you fix 400 and 500 errors?

Fixing 404 errors

For each 404 you find, you have four options:

1. Restore the page — if it was deleted by mistake, put it back.
2. 301 redirect — if the content moved, redirect the old URL to the new one permanently. This passes link equity.
3. 410 Gone — if the page had no value and you want Google to remove it quickly, return a 410 instead of 404.
4. Remove the link — if the 404 is caused by an internal link pointing to a non-existent page, update or remove that link.

Fixing 500 errors

500 errors are server-side — check your error logs immediately. Common causes and fixes:

• 500 Internal Server Error — check PHP/Node error logs. Usually a code exception, memory limit or bad configuration.
• 502 Bad Gateway — your NGINX or proxy cannot reach the upstream app. Check if your Node/PHP process is running.
• 503 Service Unavailable — server is overloaded. Add a Retry-After header during planned maintenance. Scale resources for load spikes.
• 504 Gateway Timeout — upstream is too slow. Increase proxy timeouts or optimise database queries causing slow responses.

How often should you audit for HTTP errors?

The right frequency depends on how often your site changes:

• Small sites (under 50 pages) — monthly is sufficient unless you publish frequently.
• Medium sites (50–500 pages) — after every deployment, CMS update or content migration. Weekly for active sites.
• Large sites (500+ pages) — continuous monitoring. Every structural change risks new 404s. Run a full 1,000-page audit after any major changes.
• After any site migration — always run a full audit immediately after moving to a new domain, CMS or hosting provider.

What is a false 200 status and why does it hurt SEO?

Some websites return an HTTP 200 OK status on pages that have no real content — for example an empty search results page or a CMS that serves a generic placeholder when a URL does not match any content. Google treats these as low-quality pages and may exclude them from its index.

Common scenarios where this occurs:

• A CMS that returns HTTP 200 on every URL regardless of whether content exists
• Search results pages with zero results that still return HTTP 200
• Thin pages with minimal content that do not satisfy the search intent

The fix is to ensure every URL either returns real content with HTTP 200, redirects to a relevant page with HTTP 301, or returns HTTP 404 or 410 if the content genuinely does not exist.

Related tools