Fixing Request Timeout Errors

Where a request can hang

Before a single byte of your page arrives, a request passes through several stages, and each one can stall on its own. Knowing which stage hung is most of the diagnosis, because each points to a different part of your set-up.

The last two stages overlap with other guides. A consistently slow first byte that still finishes is a speed problem covered in the TTFB guide. A proxy that waits for your application and then sends a 504 has at least answered; that is the 504 Gateway Timeout guide. This guide is about requests where nothing answers in time.

The fix, step by step

Reading a curl timing breakdown

This one command gives you the figures for step 1. Each value is the number of seconds from the start of the request to the end of that stage, so they add up as you go down.

curl -o /dev/null -s --max-time 60 -w "dns:     %{time_namelookup}\nconnect: %{time_connect}\ntls:     %{time_appconnect}\nfirst:   %{time_starttransfer}\ntotal:   %{time_total}\ncode:    %{http_code}\n" https://www.example.co.uk/services/

Four common patterns and what they tell you:

When curl gives up it prints an error such as "Operation timed out" (exit code 28) or "Could not resolve host" (exit code 6). Note which one you get, and after how long, because it narrows the stage as well as the timings do.

DNS problems that look like a dead server

DNS failures are easy to miss because they are often partial. Most resolvers cache answers, so you may see the site perfectly while visitors whose resolver asks a broken nameserver wait and give up.

• One nameserver out of several is down. Resolvers usually retry another, but only after a wait of several seconds. Some clients time out first.
• Delegation left pointing at an old provider. After moving DNS, the registrar still lists the old nameservers, which may still answer with old records or stop answering altogether.
• A records pointing at a retired server. After a hosting move, an old IP left in one record, for example on the www name but not the bare domain, sends part of your traffic to a machine that no longer serves the site.
• AAAA records with no IPv6 service. Modern browsers try IPv6 and IPv4 almost together and fall back quickly, so people may not notice. Crawlers, monitoring tools and some older clients can wait on the IPv6 address until they time out. Either make the site work over IPv6 or delete the AAAA record.

dig +short NS example.co.uk
dig +short @ns1.example-dns.net www.example.co.uk A
dig +short @ns1.example-dns.net www.example.co.uk AAAA
curl -6 -sI --max-time 15 https://www.example.co.uk/ | head -1

If you use Cloudflare or another proxying CDN, visitors connect to the CDN, not your server, so the edge addresses are what should appear in public DNS. Cloudflare reports a failed connection from the edge to your origin as a 522 error, which is covered by the same firewall and IP checks in step 3.

Keep-alive, worker slots and why busy sites stop answering

Keep-alive lets a browser reuse one connection for several requests, which saves time. The cost is that the server holds that connection open, waiting, after the last request. On servers where each open connection ties up a whole worker, long keep-alive settings can use up every worker on idle connections, and new visitors queue until they time out.

This mainly affects Apache running the prefork module, which is still common where PHP runs as an Apache module. Each connection occupies one process for as long as it is kept alive.

# Apache: check which multi-processing module is active
apachectl -V | grep -i mpm
# Apache prefork example
KeepAlive On
KeepAliveTimeout 3
MaxRequestWorkers 60

Do not simply raise MaxRequestWorkers until the warnings stop. Each prefork process with PHP loaded can use tens of megabytes, and a limit higher than memory allows leads to swapping, which makes every request slow and produces more timeouts, not fewer.

Timeouts and search engines

Search engines deal with timeouts the same way as visitors: they wait for a limited time and then record a failure. Occasional failures are retried. Repeated connection failures lead Google to slow its crawling of the site, and pages that cannot be fetched for a long period can drop out of the index until they recover.

One file deserves special attention. Crawlers request robots.txt before anything else, and Google has documented that when it cannot fetch robots.txt because of server errors or timeouts, it may hold back from crawling the site for a while. A slow or unreachable robots.txt can therefore stop crawling of pages that would have loaded perfectly well. Serve it as a static file, ideally cached at your CDN, and include it in your uptime checks.

In Search Console, open Settings and then Crawl stats. The host status panel reports problems with robots.txt fetching, DNS resolution and server connectivity over recent months. A spike in server connectivity failures that matches your audit results confirms that crawlers are meeting the same timeouts.

Preventing timeouts

• Keep DNS simple: one provider, nameservers at the registrar matching that provider, and no records left pointing at old servers.
• Publish an AAAA record only when the site genuinely works over IPv6, and test it after every server change.
• Configure security tools to reject with a status code rather than silently drop, and to recognise verified search engine crawlers.
• Use an event-driven web server set-up, with PHP-FPM or an application server behind it, rather than one process per connection.
• Cache pages at a CDN or the server so bursts of traffic do not all need a worker.
• Monitor from outside your network, record connection failures as well as error codes, and alert on both.
• Run the HTTP Error Audit on a schedule, including at busy times of day.