Robots.txt Manager

Generate a robots.txt, validate an existing one, and test specific rules against any URL — including sitemap declaration.

Implements standard RFC 9309 path matching

Directives

User-agentSitemap URL

Disallow paths (one per line)Allow paths (one per line)

Crawl-delay (seconds)

robots.txt

User-agent: *
Disallow: /admin
Disallow: /private
Allow: /