Robots.txt Generator
Generate a robots.txt file for your website with custom rules online for free.
Quick Presets
Rules
Disallow
Allow (overrides Disallow)
Crawl-delay (seconds)
Sitemap URLs
Extra / Comments
Preview
User-agent: * Disallow: /admin/ Disallow: /private/ Sitemap: https://example.com/sitemap.xml
Place this file at: https://yourdomain.com/robots.txt
حول هذه الأداة
Create a valid robots.txt file with custom allow/disallow rules for different crawlers. Add sitemap URLs, set crawl delays, and preview the generated file before downloading.
كيفية الاستخدام
Add Rules
Choose a user-agent and add Allow/Disallow paths.
Configure Sitemap
Optionally add your sitemap URL.
Download
Preview and download the robots.txt file.
حالات الاستخدام
SEO Control
Control which pages search engine bots are allowed to crawl.
الأسئلة الشائعة
Where do I put the robots.txt file?
Place it at the root of your domain: https://yourdomain.com/robots.txt