Should production code ever load files from a dev folder?

Yes, even when the files are identical today. The dev folder is where you experiment. The next time you change that helper in dev to try something, you've silently changed production too, with no deploy, no test and no record of it. It also means you can't safely delete or reset your dev copy.

Point every require at the production path, check they're byte-identical first (compare the md5 of both), and restart.

Makes sense. Is there a quick way to check nothing else is doing it?

Search the live code for the dev folder's name, excluding node_modules:

grep -rl "site-dev" /path/to/live --include=*.js --include=*.json --include=*.html

Also check your nginx config, symlinks, .env files and cron jobs for dev paths, and make sure no dev processes are still running. And watch for old page copies with canonical tags pointing at a dev subdomain, which is the same problem showing up in SEO.