Custom Robots.txt Generator

Generate a perfect robots.txt file to control how search engines crawl your website.

Default Access (All Robots):

Crawl Delay (Seconds):

XML Sitemap URL (Optional):

Googlebot:

Bingbot:

Restricted Directories (Comma separated): Directories you want to hide from search engines.

Generated Robots.txt:

What is a Robots.txt file?

A robots.txt file tells search engine crawlers which URLs the crawler can access on your site. This is used mainly to avoid overloading your site with requests or keeping private pages out of search engine results.

Where should I put this file?

For most websites (like WordPress or custom HTML), the file must be named robots.txt and placed in the root directory of your website (e.g., https://www.yoursite.com/robots.txt).

Blogger Users

If you are using Blogger, you can add these rules by going to Settings > Crawlers and indexing > Custom robots.txt.