Real Traffic Audit Guide: How It Works and How to Read It
The Real Traffic Audit answers a question analytics dashboards cannot: how many of the visits to your website came from people. It reads your web server access log, the file that records every request the server handled, and sorts each request into a group, from your own server and AI crawlers through to likely and confirmed people. This reference explains what the audit reads, the order it sorts requests in, what each optional field changes, how Google Analytics and Search Console fit in, what all eleven report sections show, and where the method reaches its limits.
What the audit reads
Every time anything asks your web server for a file, whether a person's browser, a search engine or a script, the server writes one line to its access log. The audit works on those lines. It reads the standard combined log format that Apache, nginx, Plesk and cPanel use, in which each line holds the requesting IP address, the date and time, the method and path requested, the status code returned, the size of the response, the referring page and the user agent string.
203.0.113.45 - - [14/Sep/2026:10:21:07 +0100] "GET /services/ HTTP/2.0" 200 18342 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) ..."That one line already tells you a lot: someone at that address asked for the services page, got it successfully, and arrived from Google. Multiply it by every request over a few weeks and you have a complete record of what your server did and for whom. Lines in any other format are not guessed at; they are counted as unrecognised, so you can see if part of a file was skipped.
Why a log rather than analytics
Analytics only knows about visits where a browser ran its script, and it cannot see who ran it. That works in both directions. It misses people who block or decline the script, and it counts machines that happen to run it. The tool page describes one site where analytics reported 4,600 users while the server log showed 16 people who actually used the site, with half of all page loads coming from the site's own server. A log has no script to run, so it sees every request, which is why the audit treats it as the more honest record.
How requests are sorted
Every request is sorted in this order:
| Order | Group | How it is recognised |
|---|---|---|
| 1 | Your own server | Requests from the server IP you enter: the site talking to itself. |
| 2 | AI crawlers | Named agents such as GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Amazonbot, split into training crawlers, AI search crawlers and live fetches made because a person asked an assistant. |
| 3 | Search engines | Googlebot, Bingbot, DuckDuckBot and others. |
| 4 | Headless browsers and tools | Automated browsers, speed testers and uptime monitors that identify themselves. |
| 5 | Other bots and scanners | Scrapers, link-preview bots and attack probes. |
| 6 | You and your team | IPs you list, plus any address that successfully used an admin area. |
Likely people and confirmed people
What remains after those six groups is browser-like traffic, and the audit applies two tests to it. A visitor counts as a likely person when its browser loaded a page together with the page's images, styles or scripts, and did so without hammering the site. Disguised bots tend to fetch the HTML alone, so this filter catches many of them. A confirmed person also submitted a form or used a feature, which is the strictest count. On a brochure site with no forms, the confirmed count will be small by design; that is not a fault. Read the two together: likely people is your realistic audience size, and confirmed people is the strictest, most reliable count.
Files, fields and Google connections
Files the audit accepts
Drop in Apache, nginx, Plesk or cPanel access logs as .log or .txt files,
gzipped .gz files, files with no extension such as access_ssl_log, or a .tar
or .tar.gz archive containing several logs. Several files at once is fine, so you can cover
this week and last week, or a month of rotated files, in one run. Download the access log, not the error
log.
The four fields
| Field | What it changes | If you leave it blank |
|---|---|---|
| Your domain | Links between your own pages are not counted as outside visitors in Where People Came From. | Internal clicks can look like referrals from another site. |
| Your server's IP (optional) | Separates your server's requests from visitors and fills the own-server part of the report. | Self-requests that look like browsers may end up among the people. |
| Your own IPs (optional) | Leaves you and your team out of the people counts. | Your own browsing adds to the totals unless you logged in to an admin area. |
| Analytics users, same period (optional) | Compares what your analytics claims with what your server saw. | No comparison figure, unless Google Analytics is connected. |
Your server's IP is usually shown in your hosting control panel or welcome email. Your own address is whatever a "what is my IP" search shows from your office or home connection.
Google Analytics and Search Console
Connecting is optional and needs a free aiwebpageseo account; the log analysis itself does not. You sign in on Google's own page and grant read-only access, so the audit never sees your Google password and cannot change anything in your accounts. Disconnecting revokes that access at Google straight away. Google data is only shown for websites you have verified you own with aiwebpageseo, using its domain verification; a verified Search Console property alone is not enough.
Running the audit, step by step
Step 1
Gather every access log file
Download
the access logs from your host rather than a single day's file. In Plesk they sit in the domain's top-level
logs folder as access_ssl_log, access_log and older .processed
and .gz files; in cPanel use Metrics → Raw Access; on your own server look
in /var/log/nginx or /var/log/apache2. Skip error logs and statistics files.
More days give steadier figures, so collect everything your host still keeps.
Step 2
Open the audit and drop the files in
Open the Real Traffic Audit in a desktop browser and drag all the files into the drop area, or click it to choose them. Compressed files and archives can go in exactly as downloaded, so there is no need to unpack anything first. The files are read from your computer by your own browser and are never uploaded, which is why large logs may take a little while on an older machine.
Step 3
Fill in the optional fields
Type your domain so internal links are not mistaken for referrals. Add your server's public IP address so the site's own requests are separated out, and list the addresses you and your team normally browse from. If you know how many users Google Analytics reported for the same dates the log covers, enter that number too. Each field you fill in makes the people counts cleaner and the comparison more meaningful.
Step 4
Connect Google data if you want the comparison
If you have a free aiwebpageseo account and want analytics and search figures beside your log, sign in and use Connect Google Analytics and Connect Search Console. Approve read-only access on Google's page and choose the property for this site. Google data only appears for domains you have verified you own with aiwebpageseo, through its domain verification page. Skip this step entirely if you just want the log analysis; nothing else depends on it.
Step 5
Run the analysis and check the headline
Click Analyse log. When reading finishes, the headline figure shows likely real people, meaning unique visitors in this log. Before trusting it, check that the days shown in the per-day chart match the period you meant to cover, and that few lines were counted as unrecognised. A large unrecognised count or a shorter span of days than expected usually means a file is missing or in a different format, so fix that and run it again.
Step 6
Work through the sections and save the outputs
Read the report from top to bottom: people first, then the automated traffic, then sources, pages, crawlers, security and errors, and finally the Google comparison and daily chart. Note anything that needs action. Then use Download PDF report to keep a dated copy, and Results file for the Security Audit if you want to take the security findings further. Close the tab when finished and the analysed data is discarded.
The eleven report sections
| Section | What it shows | How to read it |
|---|---|---|
| 👥 Real People | Likely and confirmed people in the log. | Your best estimate of human visitors, with the confirmed count as the strictest, most reliable count. |
| 🖥 Your Own Server & Automated Browsers | Requests from your server IP and from self-identifying headless browsers, speed testers and monitors. | If these are large, they are probably inflating your analytics. |
| 🧭 Where People Came From | Referring sources for people: search, social, AI assistants and Direct / none. | A high Direct / none share is normal; sources only count when the browser reports them. |
| 📄 What People Read | The pages likely people requested. | Your real popular content, free of crawler noise. |
| 🤖 AI Crawlers & AI Assistants | Training crawlers, AI search crawlers and live fetches. | Live fetches recurring at the same minute on several days are flagged as scheduled. |
| 🔎 Search Engine Crawlers | Googlebot, Bingbot, DuckDuckBot and others. | Shows whether and how often search engines are reading the site. |
| 🔐 Security Events | Failed logins, password-guessing bursts, admin areas reached, new accounts, attack probes and bursts of server errors. | Probes are background noise; unknown admin access and new accounts need checking. |
| ⚠ Errors Real Visitors Hit | Error responses served to people rather than bots. | Each entry is a real dead end or failure worth fixing. |
| 📊 Who Your Server Works For | Shows how your server's requests divide between people and everything else. | Puts people in proportion to everything else your hosting serves. |
| 📈 Google Analytics & Search Console vs Your Server Log | Google's figures for the same dates beside the log's. | Explains the gap between what analytics claims and what the server saw. |
| 📅 Likely People Per Day | A daily count of likely people. | Spot spikes, quiet days and the effect of campaigns. |
Reading the AI figures
The three kinds of AI traffic mean different things. Training crawlers collect pages for future models and say nothing about current visibility. AI search crawlers build the indexes assistants cite from today, so their presence matters if you want to be mentioned in answers. Live fetches show that someone asked an assistant about your page at that moment, which is the nearest the log gets to evidence of an AI recommendation. A live fetch at the same minute every day is more likely an automated scheduled task than a person.
Why Direct / none dominates
Typed and bookmarked addresses, links opened from apps and emails, and browsers that withhold where you came from all arrive without a referrer. Disguised bots add to it too. Search, social and AI-assistant visits are only counted when the browser declares its source, so treat those figures as minimums rather than totals.
Outputs and the do-it-yourself route
The report offers two downloads. The PDF report is a fixed record of the run, useful for comparing one month with the next or sharing with a developer. The results file for the Security Audit contains the security events only, for use in the Security Audit.
The tool page also documents how to do the same work by hand, without an account. It includes download
instructions for Plesk, cPanel, DirectAdmin, your own nginx or Apache server, and managed hosting; set-up
walkthroughs for Search Console and GA4; and terminal commands that read logs directly with zcat,
grep and awk. The core of the manual method is a list of bot words and a filter
for visitors that fetched a style sheet or script:
BOTS='bot|crawl|spider|slurp|curl|wget|python|headless|lighthouse|monitor|preview|scan'
zcat -f access* | grep -aviE "$BOTS" | awk '$7 ~ /\.(css|js)(\?|$)/ {print $1}' | sort -u | wc -lThe page is clear that these commands give close but rougher numbers. The audit additionally leaves out your own and your team's addresses, visitors that hammer the site, admin logins and attack probes, and lines its results up against Google Analytics and Search Console for the same dates. The manual method is a good way to check the audit's headline yourself or to automate a monthly count on a server.
How the privacy model works
Access logs record visitors' IP addresses, and under UK and EU GDPR those are personal data. Rather than asking you to trust a third party with them, the audit is built so that it never receives them. Your browser streams the file from your disk, analyses it in memory and discards it when you close the tab. Nothing is uploaded, stored or shared, including with aiwebpageseo.
One practical point follows for you: the copies of logs you downloaded are still personal data on your own computer, so delete them when you are done, and if you bundled logs on a server to download them, remove that bundle too. The Google connection is separate: it uses read-only access you grant on Google's page, and only applies if you choose to connect. None of this is legal advice; your own privacy notice should still describe how long your host keeps logs.
Limits to keep in mind
- People are counted by IP address. Several people behind one office or mobile network count once, and one person who switches from Wi-Fi to mobile data counts twice. Mobile carriers often share addresses among many customers, which pushes the count down on mobile-heavy sites.
- Perfect imitators get through. A bot that loads assets exactly like a browser can land in likely people. That is the reason the confirmed count exists.
- CDN cache is invisible. Pages served entirely from a CDN cache never reach your server, so they are missing from the log.
- One format only. Only the combined format is read; other formats become unrecognised lines.
- The log sets the window. The audit can only cover the days your files contain, and many hosts rotate logs away after a short time.
- Website builders have no logs. Wix, Squarespace, Shopify and similar platforms do not give access to server logs, so the audit cannot run on them; Search Console and Analytics are the alternatives there.
- It does not test defences. Security Events reports what happened in the log. For a proper check of your defences, use the Security Audit.
None of these make the figures wrong; they define what the figures mean. A likely people count is a well-filtered estimate of distinct addresses behaving like browsers, over the period your log covers, for requests your server handled.
1. What log format does the Real Traffic Audit need?
It reads the standard combined access-log format used on Apache, nginx, Plesk and cPanel, which includes the IP address, time, request, status, size, referrer and user agent. Lines in any other format are counted as unrecognised rather than guessed at. If you see many unrecognised lines, check whether your server uses a custom log format, or whether you downloaded an error log or a statistics file by mistake instead of the access log.
2. Can I analyse more than one log file at once?
Yes. You can drop several files in together, for example this week's log and last week's, or a set of rotated files. Gzipped files, files with no extension such as access_ssl_log, and .tar or .tar.gz archives are all accepted as they are, so there is no need to unpack or combine them first. Covering a longer period generally makes the people counts and daily chart steadier and more representative of normal traffic.
3. Why is my confirmed people count so much lower than likely people?
Confirmed people are visitors who also submitted a form or used a feature on your site, which only a fraction of any audience does. Likely people only need to have loaded a page with its images, styles or scripts without hammering the site. On a site with few forms or interactive features, the confirmed count will be low by design. Treat it as the strictest, most reliable count, and likely people as the realistic estimate.
4. Does the audit need my server IP address?
No, it is optional, but it is one of the most useful fields to fill in. Without it, requests your website makes to itself, such as a scheduled job or a rendering tool loading your pages, cannot be separated out reliably. Those requests are exactly the kind that can inflate analytics. Your hosting control panel or welcome email normally lists the server's public IP address, and that is the one to enter.
5. Why do I need an account to connect Google Analytics?
The log analysis runs entirely in your browser and needs nothing from aiwebpageseo, so it works without an account. Connecting Google Analytics and Search Console is different, because the read-only permission you grant on Google's page has to be held against an account so that the tool can fetch your figures. The account is free. Google data is only returned for websites you have verified you own with aiwebpageseo, and you can disconnect at any time.
6. What does a scheduled live fetch mean in the AI section?
Live fetches happen when an AI assistant retrieves your page because someone asked it about the page. When the audit sees such a fetch arriving at the same minute on several different days, it flags it as scheduled, because real people do not ask at an identical minute every day. It is more likely an automated task set up in an assistant or a monitoring service. Count unscheduled live fetches as the stronger sign of genuine interest.
7. Can the audit run on a Shopify or Wix website?
Not directly. Hosted website builders such as Wix, Squarespace and Shopify do not give their customers access to raw server access logs, and the audit needs those logs to work. On those platforms, use Google Search Console for search impressions and clicks and Google Analytics for visitor behaviour instead. If you move to hosting where you control the server, or your host can provide raw logs in combined format, the audit becomes available.
Real Traffic Audit Fixes takes each finding in turn: your own server and bots inflating analytics, AI crawler decisions, errors real visitors hit, security events, short or partial logs and analytics set-up problems.
Open Real Traffic Audit Fixes →