Robots.txt Generator
Generate a robots.txt file for your website with custom rules online for free.
Quick Presets
Rules
Disallow
Allow (overrides Disallow)
Crawl-delay (seconds)
Sitemap URLs
Extra / Comments
Preview
User-agent: * Disallow: /admin/ Disallow: /private/ Sitemap: https://example.com/sitemap.xml
Place this file at: https://yourdomain.com/robots.txt
Informazioni su questo strumento
Create a valid robots.txt file with custom allow/disallow rules for different crawlers. Add sitemap URLs, set crawl delays, and preview the generated file before downloading.
Come usare
Add Rules
Choose a user-agent and add Allow/Disallow paths.
Configure Sitemap
Optionally add your sitemap URL.
Download
Preview and download the robots.txt file.
Casi d'uso
SEO Control
Control which pages search engine bots are allowed to crawl.
Domande frequenti
Where do I put the robots.txt file?
Place it at the root of your domain: https://yourdomain.com/robots.txt