Robots.txt Generator

Search Engine Optimization

Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

A Robots.txt file is present at the root of your website and it protects those parts of the website that you don’t want Search Engine Crawlers to have an access to. Search Engine Crawlers or web crawlers are those programs that browse the internet in a methodical way to provide updated data to a specific search engine.

Robots.txt is very important as in the absence of this; web crawlers will have access to the hidden information. Also with a wrong Robots.txt file, your webpage will not appear in the search results. This can result in the downfall of the search engine ranking of your site. Our “Robots.txt Generator” tool will simplify your task and generate an efficient Robots.txt file, which will safeguard your website.

To use this you will have to choose the entry of all robots and the crawl-delay you prefer. Apart from this, fill the sitemap, search engines, and restricted directories. After this, your robots.txt will be generated. Robots.txt Generator is a crucial tool for the performance of your website.


Free Photos