Robots.txt Generator
Generate a robots.txt file for your website with custom rules online for free.
Quick Presets
Rules
Disallow
Allow (overrides Disallow)
Crawl-delay (seconds)
Sitemap URLs
Extra / Comments
Preview
User-agent: * Disallow: /admin/ Disallow: /private/ Sitemap: https://example.com/sitemap.xml
Place this file at: https://yourdomain.com/robots.txt
이 도구에 대하여
Create a valid robots.txt file with custom allow/disallow rules for different crawlers. Add sitemap URLs, set crawl delays, and preview the generated file before downloading.
사용 방법
Add Rules
Choose a user-agent and add Allow/Disallow paths.
Configure Sitemap
Optionally add your sitemap URL.
Download
Preview and download the robots.txt file.
활용 사례
SEO Control
Control which pages search engine bots are allowed to crawl.
자주 묻는 질문
Where do I put the robots.txt file?
Place it at the root of your domain: https://yourdomain.com/robots.txt