Robots.txt Generator
Generate robots.txt file.
Rules
Custom Rules
robots.txt Output
Frequently Asked Questions
Where do I put my robots.txt file?
Upload it to the root of your domain, e.g. https://www.example.com/robots.txt. It won't work in a subfolder.
Does robots.txt guarantee pages won't be indexed?
No. It only asks well-behaved crawlers not to access certain paths. Use a noindex meta tag if you need to prevent indexing.