# Free AI Crawler Robots.txt Generator & Checker

> Browser-only utility for generating and auditing AI crawler robots.txt rules.

URL: https://ai-crawler-robots-builder.vercel.app/
Source: https://github.com/Turner-Levey/ai-crawler-robots-builder
License: MIT
Builder: Turner Levey
Tiny Tool Town listing: https://www.tinytooltown.com/tools/ai-crawler-robots-txt-builder/
No-Login pending listing: https://nologin.tools/tool/ai-crawler-robots-builder-vercel-app/
No-Login badge status: https://nologin.tools/badge/ai-crawler-robots-builder-vercel-app/

## Useful Paths

- https://ai-crawler-robots-builder.vercel.app/ - interactive AI crawler robots.txt generator and checker
- https://ai-crawler-robots-builder.vercel.app/sitemap.xml - sitemap
- https://ai-crawler-robots-builder.vercel.app/robots.txt - robots policy

## Notes

- No tracking, cookies, analytics, beacons, API calls, or external scripts.
- Presets cover OAI-SearchBot, GPTBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, Google-Extended, and PerplexityBot.
- Search intent: free AI crawler robots.txt generator, robots.txt checker for AI crawlers, GPTBot robots.txt rules, ClaudeBot robots.txt rules, PerplexityBot robots.txt rules.
- Source docs refreshed 2026-05-14 after Perplexity's public crawler docs moved to https://docs.perplexity.ai/guides/bots.
- Official crawler docs should be checked before publishing production rules.