Free guide:winning new clients predictably in 2026 · 10 pages, freeGet it now
← back to glossary
Discovery

Crawl budget

The number of pages a search engine fetches from a site in a given period - a topic for very large websites and for almost nobody else.

What is Crawl budget?

Crawl budget is not an account you fill or drain. Google describes it as the interplay of two quantities: how many requests a server tolerates, and how much demand there is for the pages. Demand, per Google's own documentation, depends on the size, update frequency, quality and relevance of the site, plus the popularity of individual URLs and how quickly a stored copy goes stale. That is a demand model, not an editorial calendar.

The same guidance states plainly who it is written for: sites from one million pages with weekly changes, or from ten thousand pages with content changing daily. An ordinary company website sits two to three orders of magnitude below that. For such a site crawl budget is not a lever but a distraction from questions that would actually change something.

What does help small sites is not budget optimisation but tidying: no endless filter and sort URLs, no redirect chains, a correct sitemap, sound internal linking. It has the pleasant side effect of being the same work that makes pages easier for visitors to find. And the Search Console button that requests indexing for a page works exactly once: pressing it repeatedly speeds up nothing, on Google's own statement.

Why does Crawl budget matter?

Crawl budget is the most common reason small websites schedule work that cannot possibly help them. Google draws the line itself and draws it high. Sites below it can drop the topic entirely and spend the time on the pages that carry a wrong figure.

Crawl budget in practice

  1. 01A site with 60 URLs schedules a crawl budget audit and cancels it after reading the thresholds Google names.
  2. 02A shop with a six-figure URL count blocks filter combinations that consume most of the fetches without ever producing visibility.
  3. 03A team submits the same URL for indexing every day and stops after reading Google's note on repeat submissions.

Related terms

These topics are your day-to-day?

We build marketing systems along exactly these disciplines - in strategy, execution and tech integration.

Your project as the next case.

We build marketing not as a service, but as a system. Let's start with a conversation.

30 min. · free · 24 h reply · no agency deck