Metric v1.0, updated 2026-07-07

What crawl efficiency is

The share of the URLs a crawler fetched on your site that can actually be indexed, and why the rest is wasted budget.

Crawl efficiency is the share of the URLs a crawler fetched on your site that can actually be indexed. If a bot fetches a hundred URLs and eighty of them are real, indexable pages, your crawl efficiency is eighty per cent. The other twenty were budget spent for nothing.

How to read it

Read it as a percentage, higher is better, where a hundred per cent means every URL a crawler spent a fetch on was a page it could index. Topkay shows it next to the count it is built from, so many of so many fetched URLs cannot be indexed, and lists what the wasted fetches were. Treat a number well below a hundred as a sign the crawler is being sent to the wrong places, not as a verdict on the quality of any one page.

Why it matters

A crawler gives your site a finite amount of attention, its crawl budget. Every fetch it spends on a redirect, an error page, a noindex page or a duplicate is a fetch it does not spend on a real page. Low efficiency means your best content is competing for whatever attention is left after the waste, and on a large site some of it may never be reached at all.

What moves it

The waste is made of the same handful of things, so the fixes are too. Collapse redirect chains and point internal links at the final URL so nothing has to redirect. Fix or remove links into error pages. Keep noindex pages out of a crawler's path. Resolve duplicates with a clean canonical, and strip tracking parameters that turn one page into many addresses. See trim crawl waste for the step by step, and run the Audit to see where your waste is.

It is not the proprietary weighting behind your score, and it is not a promise. It is a plain read on how much of a crawler's effort your site lets it use. Raise it by removing the waste, not by hiding pages.

Read next

What is ARMX

Agent Readiness and Machine eXperience: the five things every machine does with your site, and the one score that measures them.

The origin and its many viewpoints

One source, read differently by browsers, crawlers, AI agents, and a growing list of tools, plugins and apps.

Core Web Vitals, plainly

The three numbers that measure how a page feels to a person.

All topics
Ask Topkay the analyst×
Ask me anything about your site. I can run Topkay's tools on your twin (the audit, page speed, headers, schema, AI access), crawl or refresh it, and generate the fixes you are missing, all from here. I read the Learn corpus too and cite both. Try a skill below.