Fixing Lost URLs After a Site Migration
A lost URL is an address that worked on your old site and now returns an error, lands on the wrong page or quietly bounces to the home page. Every one of them can carry links from other websites, bookmarks, old emails and search history, and all of that value stops at the error.
Owners usually notice when traffic to a product, service or article page vanishes after launch. This guide shows how to build a complete list of the old addresses, decide the right outcome for each one, and put a redirect map live that you can prove is complete.
What counts as a lost URL
The Site Migration Audit compares the pages found in a crawl from before the move with the live site. Anything that was reachable before and is not reachable in the same way now is a lost URL. In practice that covers more situations than a plain 404:
| What the old address does now | Why it counts as lost |
|---|---|
| Returns 404 Not Found | The most obvious case. Links and history pointing at it lead nowhere. |
| Redirects to the home page | Google may treat a redirect to an unrelated page as a soft 404, so the old page's signals are not passed on. Visitors also lose the thing they came for. |
| Redirects to a loosely related category | Better than the home page, but if the specific product or article still exists elsewhere, the match is wrong. |
| Returns 200 but shows a "not found" message | A soft 404. The server says the page is fine while the content says it is gone. |
| Returns 500, 502 or 503 | Often a new platform failing on an old URL pattern, such as a legacy query string it cannot parse. |
| Works only with a different case or slash | For example /Services/Roofing
worked before and only /services/roofing/ works now. |
The last row catches many people out. Windows-hosted and some older platforms treated paths as case-insensitive,
while Linux servers and most modern platforms treat /About and /about as different
addresses. Links from other sites keep whatever capitalisation was used when they were written.
Where the old addresses come from
No single source lists every URL your old site had. A crawl only finds pages that were linked internally, so orphaned landing pages, old campaign pages and files uploaded years ago are missing from it. Combine several sources and remove duplicates.
- A crawl of the old site. The best starting point if you made one before launch. The Site Migration Audit uses this pre-migration crawl as its baseline.
- Search Console performance data. Export the Pages tab for the last 16 months from the property that covered the old site. Every URL with an impression is one Google knew about.
- Search Console links report. The Top linked pages list shows addresses other sites point at. These matter even if they had little traffic.
- Analytics landing pages. Landing-page reports show addresses real visitors arrived on, including email and paid campaign pages that were never in the navigation.
- Server access logs. Logs from the old server, and from the new server since launch, show every requested path. Filter the new logs for 404 responses to see which old addresses people and crawlers are still asking for.
- Old XML sitemaps. Check backups, the old hosting account, or a copy still cached on a CDN.
- The Wayback Machine. Its URL listing for your domain can reveal sections nobody remembers.
Put everything in one spreadsheet column, normalise the protocol and host so that http://,
https://, www and non-www versions of the same path collapse to one row, and
keep the original form in a second column so you can test exactly what was linked.
Restore, redirect or retire
Not every lost URL should be redirected. The right outcome depends on whether the content still exists, whether a close equivalent exists, and whether anyone still wants it.
| Situation | Outcome | Status code |
|---|---|---|
| The content was accidentally left out of the new site | Restore the page, ideally at a new address that follows the new structure, then redirect the old one to it | 301 or 308 from old to new |
| The same content lives at a new address | Redirect to it | 301 or 308 |
| A product was replaced by a newer model | Redirect to the successor if it genuinely serves the same need | 301 or 308 |
| Several thin pages were merged into one guide | Redirect each to the merged page, to the relevant heading if you like | 301 or 308 |
| The content is gone with no equivalent, but it has links | Consider rebuilding a useful page on the topic; otherwise let it go | 404 or 410 |
| The content is gone, no equivalent, no links, no traffic | Let it go | 404 or 410 |
| Junk addresses: old search results, session IDs, spam | Let them go | 404 or 410 |
A 410 Gone tells crawlers the removal was deliberate. Google treats 404 and 410 very similarly in practice, so either is fine. What matters is that the response is an honest error with a helpful page, not a redirect to somewhere that does not answer the visitor's question.
Both permanent redirects (301, 308) and temporary ones (302, 307) pass signals in Google's systems. For a migration, use a permanent redirect anyway: it states your intention clearly, and other search engines and tools read it the same way.
Matching old addresses to new ones
Matching is where most of the effort goes. Work from the most valuable rows downwards and use the approach that fits the pattern of change.
Pattern changes
When a whole section moved in a predictable way, such as /blog/2019/05/slug/ becoming
/articles/slug/, one pattern rule can cover hundreds of rows. Write the rule, then still
test every row, because patterns always have exceptions: two old posts with the same slug in different
years, or a slug that was changed during the rebuild.
One-to-one changes
When slugs changed individually, match on page title or main heading. Export titles from the old crawl and the new crawl, then use a lookup in the spreadsheet to pair exact matches. Review near matches by hand.
No obvious match
Search the new site for the old page's main topic, look at which queries the old page ranked for in Search Console, and choose the new page that answers those same queries. If nothing does, the row belongs in the retire group or needs new content.
old_url,old_title,clicks_16m,linking_domains,decision,new_url,tested_status,notes
/services/Roofing-Repairs.aspx,Roofing repairs Leeds,812,14,redirect,/services/roof-repairs/,301,case + extension changed
/blog/2019/05/gutter-guide/,Gutter cleaning guide,95,3,redirect,/guides/gutter-cleaning/,301,
/downloads/price-list-2021.pdf,Price list 2021,0,6,redirect,/pricing/,301,linked from trade directory
/offers/spring-2020/,Spring 2020 offer,0,0,retire,,410,expired campaignKeep the tested status column. It is the proof that the map is not just written but live, and it is the column you re-check after every deployment.
Where the rules go on common platforms
- Apache.
Redirect 301orRewriteRulelines in the virtual host or.htaccess. UseRewriteMapfor thousands of one-to-one rows rather than thousands of separate rules. - Nginx. A
mapblock that looks up the old path and returns the new one, then a singlereturn 301when a match is found. - WordPress. A redirect plugin or the redirect module of your SEO plugin, or server rules for large maps. The WordPress migration guide covers permalink changes in particular.
- Shopify. URL redirects can be imported in bulk from a CSV, and only take effect when no live page exists at the old path. See the Shopify migration guide.
- CDN or edge. Cloudflare and similar services can run redirect lists before requests reach the server, which keeps the rules in one place when several back ends sit behind one domain.
Wherever the rules live, match the query string deliberately. Old platforms often used addresses such
as /product.php?id=42, and many redirect tools ignore the query string unless told otherwise,
which sends every product to the same destination.
The fix, step by step
Step 1
Rebuild the list of old addresses
Combine the pre-migration crawl, the Search Console Pages export from the old property, the top linked pages list, analytics landing pages, old sitemaps and 404 lines from the server logs into one spreadsheet column. Normalise protocol and host so duplicates collapse, but keep the original form of each address in a second column so you can test exactly what other sites and visitors actually used.
Step 2
Test every old address against the live site
Run the Site Migration Audit, which compares the pre-migration crawl with the live site and lists lost URLs, then add the extra addresses from your other sources. Record the first status code and the final destination for every row. Flag anything that ends on the home page, on a generic category, on a 200 page that says not found, or on a server error, as well as plain 404s.
Step 3
Rank the failures by what they carried
Add columns for clicks in the last 16 months and for linking domains, both taken from Search Console or a backlink tool, then sort by those figures. The top of the list is where almost all the recoverable value sits, so fix those rows first and by hand. Low-value rows at the bottom can be handled with pattern rules or left as honest 404 responses later.
Step 4
Choose restore, redirect or retire for each row
Write a decision in its own column. Restore content that was left out by mistake and still has an audience. Redirect when the same or clearly equivalent content lives at a new address. Retire with a 404 or 410 when the content is gone and nothing on the new site answers the same need. Never choose the home page as a fallback destination just to make an error disappear.
Step 5
Match each redirect to its true equivalent
For predictable section moves write a pattern rule and list the exceptions. For individual slug changes, pair old and new pages by title using a spreadsheet lookup, then check near matches by hand. Where nothing matches, look at which queries the old page ranked for and pick the new page that answers them. Watch for query strings, uppercase letters, file extensions and trailing slashes in the old addresses.
Step 6
Load the rules and retest the whole list
Add the redirects at the server, platform or CDN, using permanent 301 or 308 codes, and purge any cache. Then test every row in the spreadsheet again rather than a sample, and fill in the tested status column. Each redirected address should reach its destination in a single hop with a final 200, and each retired address should return a real 404 or 410 with a helpful page.
Step 7
Watch for stragglers for the next few months
Check the Not found (404) group in the Search Console Page indexing report and filter the new server logs for 404 responses every week at first. Addresses you never knew about will appear as crawlers and visitors follow old links. Add each worthwhile one to the map, retest, and re-run the Site Migration Audit so the lost URL count keeps falling instead of drifting back up.
Edge cases that hide lost URLs
The redirect works in a browser but not for crawlers
Some redirects are done with JavaScript or a meta refresh after the page loads. Google can follow JavaScript redirects when it renders a page, but a server-side redirect is faster and more reliable, and many other crawlers and link tools only see server responses. Test with a tool that shows the raw status code, not just the address bar.
Redirects that depend on the visitor
Geo-redirects, language detection and cookie banners that redirect first-time visitors can send crawlers somewhere different from people. Make sure an old address always gives the same answer whoever asks.
Old addresses with fragments
Anything after a # is never sent to the server, so you cannot redirect /faq#delivery
separately from /faq. Redirect the page and make sure the destination has a heading with
the same id if you want old fragment links to land in the right place.
Mixed old hosts
If the old site answered on several hosts, such as shop.example.co.uk, example.co.uk
and a forgotten old.example.co.uk, every host needs its own redirects and a working certificate.
A redirect cannot run if the browser stops at a certificate warning first.
Preventing lost URLs next time
- Freeze a record before launch. Crawl the old site, export Search Console and analytics data, and save the old sitemap before anything changes. This is the baseline the Site Migration Audit needs.
- Make the map a launch requirement. The redirect spreadsheet should be finished and tested on staging before the new site goes live, not written afterwards from the error reports.
- Keep URLs that do not need to change. The simplest lost URL to fix is one that never moved. Change paths only when there is a real reason.
- Keep redirects long term. Google recommends keeping site-move redirects for at least a year, and old links elsewhere never stop being clicked.
- Store the map with the code. Keep the redirect list in version control so a future redesign starts from it instead of overwriting it.
1. How do I find URLs that were lost in a migration if I never crawled the old site?
Rebuild the list from other sources. The Search Console performance report for the old property keeps up to 16 months of page data, the links report lists pages other sites point at, and analytics landing-page reports show where visitors arrived. Add old XML sitemaps from backups, 404 responses in the new server logs, and the Wayback Machine listing for your domain. Together they usually recover the addresses that actually mattered, even without a crawl.
2. Is it better to redirect a lost page to the home page or let it return 404?
Let it return 404 or 410 unless there is a genuinely equivalent page. Google may treat a redirect to an unrelated page such as the home page as a soft 404, so the old page's value is not passed on anyway, and visitors land somewhere that does not answer their question. An honest error page with a search box and links to your main sections is more useful to people and clearer to search engines.
3. Should I use 404 or 410 for pages I am deliberately removing?
Either works. A 410 Gone says the removal was deliberate, while a 404 Not Found just says nothing is there. Google has said it treats the two very similarly, and any difference in how quickly a URL drops out is small. Choose whichever your platform makes easiest to set. The important part is that the response is a real error code and not a 200 page that only says the content is missing.
4. Do 302 redirects lose value compared with 301 redirects?
In Google's systems both permanent and temporary redirects pass signals to the destination, so a 302 is not a disaster. For a migration you should still use 301 or 308, because they state that the move is permanent and help Google choose the new address as the one to show. Other search engines and many SEO tools also read the permanent code as the clearer instruction, so it avoids confusion later.
5. How many old URLs do I need to redirect?
Every old address that carried value: traffic, links from other sites, or a real audience. On most sites that is a minority of all the URLs ever created, and sorting by clicks and linking domains shows you where the line is. Pattern rules can cover the rest of a moved section cheaply. Old search result pages, session parameters and spam addresses do not need redirects and are better left as errors.
6. Why do old URLs still appear in Search Console weeks after I added redirects?
Google recrawls old addresses on its own schedule, and it keeps reporting on them while it processes the move. Seeing them in the Page with redirect group is expected and shows the redirect is being seen. They only need action if they sit in the Not found or Server error groups, or if the URL Inspection tool shows the redirect landing somewhere other than the destination you set in your map.
7. Can I redirect old PDFs and image files as well as pages?
Yes. Any address the server receives can be redirected, including PDFs, images and downloads. It is worth doing for files that other sites linked to, such as price lists, brochures, reports or infographics, because those links carry the same value as links to pages. Redirect each file to its replacement file or to the page that now holds the same information, and test that the destination loads without a login.
The Site Migration Audit compares your pre-migration crawl with the live site, finds lost URLs, validates redirects, tracks ranking shifts, checks indexing and structured data, and audits internal links.
Open Migration Audit Fixes →