robots.txt Generator
Generate a robots.txt file from common presets.
robots.txt Generator creates Allow-all, Block-all, or standard-site robots.txt files with an optional Sitemap line. Copy the output to your site root.
Frequently Asked Questions
Does my data leave the browser?
This tool runs in your browser. Nothing is uploaded to ToolWorld servers.
Does robots.txt hide private pages?
No. It asks well-behaved crawlers not to visit paths. It is not access control.
Where do I put the file?
Serve it at https://your-domain/robots.txt on the site root.