# What is Crawl Budget? | Yungle glossary

> Crawl budget is the number of pages a search engine will crawl on a site within a given time.

Source: https://yungle.ai/glossary/crawl-budget/

1.  [Resources](https://yungle.ai/resources)
2.  Glossary

Glossary · C

# Crawl Budget

In plain words

Crawl budget is the limited number of pages a search engine's bots will crawl on a given site within a period of time. Larger sites need to manage crawl budget carefully, using clean site architecture and robots.txt, so search engines spend their crawl allowance on the most important pages rather than low-value duplicates.

Example

A site with 50,000 filtered URLs makes Google waste crawls on duplicates. Blocking the filter parameters frees that budget for the pages that matter.

Related resources

-   Glossary

    ## Robots.txt

    Robots.txt is a file that tells search engine crawlers which parts of a site they can or can't access.

    [See term](https://yungle.ai/glossary/robots-txt)
-   FAQ

    ## How Much Should I Budget for Digital Marketing Monthly?

    Most growing businesses budget 7-10% of revenue for marketing overall, with website and SEO typically making up $2,000 to $8,000+ per month depending on growth goals and competitiveness. Budget should scale with ambition, not just company size. A business trying to double revenue in a year needs a heavier marketing investment than one maintaining steady growth. Track spend against pipeline and revenue generated, not just activity, to know if the budget is right.

    [See answer](https://yungle.ai/resources?type=faq#how-much-should-i-budget-for-digital-marketing-monthly)
-   Glossary

    ## XML Sitemap

    An XML sitemap is a file listing a site's pages to help search engines find and crawl them efficiently.

    [See term](https://yungle.ai/glossary/xml-sitemap)
-   FAQ

    ## How Do I Optimize My Website for Search Engines?

    Start with clean technical SEO — fast load times, proper heading structure, mobile responsiveness — then build content around what your actual customers are searching for. Most sites over-invest in keywords and under-invest in structure. Google needs to crawl and understand your site before content quality even matters. Fix the technical foundation first, then focus content on answering real customer questions clearly.

    [See answer](https://yungle.ai/resources?type=faq#how-do-i-optimize-my-website-for-search-engines)

Next step

## Let's talk

_Image: Engraving of a jungle panorama with mountains_

Inspired by Tulum, QROO, MX