Fixing 404 Not Found Errors

What a 404 actually tells you

A 404 is the server saying "I understood the request, but there is nothing here". It says nothing about why. The page might never have existed, might have been deleted last year, or might exist at a slightly different address because a slug was edited. The code is the same in every case, so the status alone cannot tell you what to do. What decides the fix is the history of the URL and who is still asking for it.

It helps to separate the three groups of people who hit a 404:

• Visitors following your own links. A menu item, a related-posts block or a link in an old article still points at a dead address. These are entirely under your control and are always worth fixing.
• Visitors and crawlers following other people's links. Another site, a newsletter, a forum post or a bookmark points at a URL you removed. You cannot edit the link, so the fix is on your side: a redirect, or bringing the page back.
• Bots guessing. Vulnerability scanners ask for /wp-login.php, /.env or /phpmyadmin/ on sites that have none of those. These 404s are correct and should be left alone.

404 compared with 410 and soft 404

Google has said for years that 404s for pages that really are gone do not harm the rest of a site. The cost is local: the visitor who hit the dead end, and any value the old URL had built up from links pointing at it.

Finding every 404, not just the ones you know about

No single source shows the whole picture. Each one sees a different slice, and you want them combined before you decide anything.

The access log is the most honest source. On a typical nginx or Apache log in the combined format, this pulls the most requested missing paths over the period the log covers:

awk '$9 == 404 {print $7}' access.log | sort | uniq -c | sort -rn | head -50

To see which pages and sites are sending people to those dead paths, include the referrer field:

awk -F'"' '$3 ~ / 404 / {print $2 " | " $4}' access.log | sort | uniq -c | sort -rn | head -50

Filter out obvious scanner paths before you count. Anything containing wp-admin on a site that is not WordPress, .env, .git, xmlrpc.php or cgi-bin is almost always automated probing and needs no redirect.

The fix, step by step

Choosing between redirect, restore and retire

The decision in step 3 is where most 404 clean-ups go wrong, usually by redirecting too much. A redirect is a promise that the new page gives the visitor what the old URL offered. When it does not, the visitor bounces and search engines tend to ignore the redirect.

When you are unsure, ask what someone arriving from the old link was trying to do, and whether the target page lets them do it within a few seconds. If the honest answer is no, a clear 404 page with good navigation serves them better than a redirect to something unrelated.

Writing redirects on common set-ups

Put redirects as close to the front of the stack as your set-up allows, and keep them in one place so the next person can find them.

Apache (.htaccess or virtual host)

# One page moved
Redirect 301 /services/boiler-repair /services/boiler-servicing
# A whole folder renamed, keeping the rest of the path
RedirectMatch 301 ^/blog/news/(.*)$ /news/$1
# Retired on purpose
Redirect gone /offers/summer-2024

nginx (server block)

location = /services/boiler-repair { return 301 /services/boiler-servicing; }
location ^~ /blog/news/ { rewrite ^/blog/news/(.*)$ /news/$1 permanent; }
location = /offers/summer-2024 { return 410; }

For long lists in nginx, a map block keeps hundreds of one-to-one redirects readable and fast:

map $uri $moved_to {
    default "";
    /old-range/oak-table   /furniture/oak-dining-table;
    /old-range/oak-chairs  /furniture/oak-dining-chairs;
}
server {
    if ($moved_to) { return 301 $moved_to; }
}

WordPress, Shopify and hosted builders

In WordPress, a redirect manager plugin or your SEO plugin's redirect feature stores rules in the database and can log 404s as they happen. WordPress also guesses at close matches for some missing URLs, which can hide typos; the WordPress HTTP errors guide covers the platform's own 404 quirks, such as rewrite rules that need flushing. Shopify has URL redirects under Online Store, Navigation (the exact menu location has moved over the years), and can create a redirect automatically when you change a product handle if you leave that option ticked. Squarespace, Wix and similar builders each have a redirects panel in site settings.

For detailed advice on avoiding multi-hop redirects once the rules pile up, see fixing redirect chains.

Soft 404s: the errors that hide behind a 200

A soft 404 is a page that looks like an error or an empty shell to a reader but returns 200 OK. Search Console reports these separately under "Soft 404", and the HTTP Error Audit will not flag them as 404s because the code is a success. They are common, and they usually come from templates rather than individual pages.

• Custom error templates that forget the status. A CMS or framework renders a friendly "page not found" design but never sets the code. Check with curl -I on a made-up path.
• Empty listings. Category, tag or search pages with no results, or filter combinations that match nothing.
• Out-of-stock or withdrawn products reduced to a title and a "no longer available" line.
• Single-page applications that serve the same HTML shell with 200 for every path and draw "not found" in JavaScript. The server or framework has to return 404 for unknown routes; the Node.js and Next.js guide shows how.
• Redirects to irrelevant pages, such as every old product sent to the homepage, which search engines may class as soft 404s.

The fix is to decide which it really is. If the page has no useful content, return 404 or 410. If it should exist, give it real content: an empty category could explain what will appear there and link to similar ranges, and a withdrawn product could name its replacement and link to it.

After a redesign, migration or platform change

The biggest wave of 404s nearly always follows a change of URL structure: a new CMS, a move from /index.php?id=12 style addresses to readable slugs, dropping date folders from blog URLs, or merging two sites. The clean-up is much easier if you plan it before launch rather than after.

1. Before the old site is switched off, export every URL it served, from the old sitemap, a full crawl and the last few months of access logs.
2. Map each old URL to its new home in a two-column sheet. Mark anything with no equivalent as "retire".
3. Load the map into your redirect rules and test the whole list against a staging copy before launch.
4. After launch, recrawl the old list and check every row lands on a 200 in one hop.
5. Watch Search Console and the access log closely for the next few weeks and add rules for anything you missed.

Keep migration redirects in place for a long time. Google has suggested at least a year, and there is little reason to remove a working redirect while old links still point at it. The migration redirects guide goes deeper into building and testing the map.

Stopping new 404s appearing

• Treat slugs as permanent. Once a page is published and shared, change its title freely but leave its URL alone unless there is a strong reason.
• Redirect at the moment of change. Make adding a redirect part of renaming or deleting any page, not a clean-up job months later. Many CMSs can prompt for this.
• Link from templates with generated URLs, not typed-in addresses, so a slug change updates menus and related links automatically.
• Keep the sitemap generated, never hand-edited, so it cannot list deleted pages.
• Log 404s continuously. A weekly look at the top missing paths in the access log, or a 404 log in your CMS, catches new problems while they are small.
• Run the audit on a schedule and after every release that touches navigation, templates or URL rules.