WordPress plugin · Free on WordPress.org
See how Google, Bing and AI crawlers really use your site.
CrawlBoost reads your server logs and shows you what every bot actually did — what it fetched, what it wasted time on, and where your crawl budget is going. It works alongside any SEO plugin and never modifies your site.
CrawlBoost — Log analysis
Crawl budget is invisible until it starts costing you.
You will not see it in a rank tracker. You see it as new pages taking three weeks to appear, and updated pages still showing the old title in search results a month later.
WordPress generates junk by default
Tag archives, author pages, date archives, attachment pages, paginated comment threads. Nobody asked for them, nobody searches for them, and Googlebot crawls all of them.
Filters multiply URLs
One product page with three filter options becomes hundreds of crawlable addresses, each looking like a separate page to a crawler that does not know better.
Your SEO plugin cannot see it
Yoast, Rank Math and SEOPress write metadata. None of them reads your access log, so none of them can tell you which bots came, what they took, or what they never reached.
What CrawlBoost does
Six diagnostic tools that read your logs and your site — never touching your sitemap, robots.txt, canonicals or meta robots.
Crawler log analyser
Upload an Apache or Nginx access log, plain or gzipped. Parsed on your own server. Shows which bots crawled you, what they fetched most, the status codes they were served, and where the budget went.
Page inventory health
Scans every published page and flags what is empty, thin or set to noindex, with edit links so you can act. Understands WPBakery, Elementor, Divi, GoodLayers and other page builders.
Status integrity
Finds soft-404 risks — pages returning 200 that look missing to a crawler — and audits your sitemap in-process, without a single loopback request that could stall on shared hosting.
URL waste radar
Detects wasteful internal links — tracking parameters, feeds, add-to-cart URLs, REST endpoints, trackbacks — and names the plugin, theme or setting that owns each fix.
HTTP 304 validator
Checks whether your homepage returns 304 Not Modified on revalidation. When it does not, crawlers re-download the full page every visit — a quiet drain most sites never notice.
IndexNow notifications
Tells Bing, Yandex, Seznam, Naver and Yep the moment you publish, update or delete a page. Switches itself off when your SEO plugin already does this, so nothing is submitted twice.
It reports. It does not rewrite.
CrawlBoost never writes your sitemap, robots.txt, canonical tags or meta robots directives. That territory belongs to your SEO plugin, and two plugins fighting over it is how sites get deindexed. Every finding shows the tested URL, the response, why it may matter, and which system owns the fix — your SEO plugin, your theme, your host or your server.
Three steps to a clearer picture
No files to edit by hand, and nothing that requires server access beyond downloading a log.
Download your access log
Most hosts expose it under Raw Access in cPanel, or Access Logs in your host panel. Apache and Nginx combined format, plain or gzipped.
Upload and parse
Parsing happens on your own server — the file never leaves it, which matters because access logs contain IP addresses and are personal data under GDPR.
Act on what it names
Run the five on-site checks alongside the log report, and each finding tells you which system owns the fix rather than leaving you to guess.
€0
Every diagnostic module, no locked features, no site limit, no account to create and no upgrade prompts in your admin. CrawlBoost came out of our own client work and it earns its keep by being installed.
- Crawler log analyser
- Page inventory health
- Status integrity & soft-404 detection
- URL waste radar
- HTTP 304 validator
- IndexNow notifications
Questions
I already have SEOPress, Yoast or Rank Math. Does CrawlBoost conflict with them?
No. CrawlBoost is read-only by design — it never touches your sitemap, robots.txt, canonical tags or meta robots. It detects which SEO plugin you are running and works alongside it, adding the diagnostic layer none of them provide.
Where do I find my server log to upload?
Most hosts expose it in cPanel under Raw Access, or in your host panel under Access Logs. CrawlBoost reads Apache and Nginx combined-format logs, plain or gzipped. The file never leaves your server — parsing happens locally.
Can CrawlBoost break my site or my rankings?
No. It reads, measures and reports. It does not modify any file, header or database content that affects how crawlers see your site. The worst it can do is show you an issue you did not know existed.
Does it work with page builders like Elementor, WPBakery or GoodLayers?
Yes. Page inventory and soft-404 detection read the WordPress render pipeline plus builder meta signatures, so builder-driven pages are not falsely flagged as empty.
Does it need an AI API key?
No. CrawlBoost does no AI analysis and contacts no AI provider. The only external request it ever sends is to IndexNow, and only if you switch that on yourself.
How large a log file can it handle?
Parsing runs inside a normal PHP execution window on your own server, so the ceiling is whatever your host allows — typically a few hundred megabytes of combined-format log. If a file times out, split it by date range and upload the parts separately.
Not sure crawl budget is your bottleneck?
Run a free site analysis and we will tell you what is actually slowing indexing down — including when the answer is something else entirely.
