Aback Audit is a website auditing service. The crawler identifies itself with this user agent:
AbackAuditBot/2.0 (+https://abackaudit.com/bot)GET and HEAD requests.Add the following to your robots.txt:
User-agent: AbackAuditBot
Disallow: /The service refuses to audit a site that asks it to stop. The request is rejected before any page is read.
| Bot | Class | Vendor | Purpose |
|---|---|---|---|
GPTBot | training | OpenAI | Collects public web content for OpenAI model training. |
ClaudeBot | training | Anthropic | Collects public web content for Anthropic model training. |
CCBot | training | Common Crawl | Open crawling dataset used by many model builders. |
Meta-ExternalAgent | training | Meta | Collects public web content for Meta AI models. |
Bytespider | training | ByteDance | Collects public web content for ByteDance models. |
Google-Extended | training | Robots.txt control token that opts a site out of Gemini training. | |
Applebot-Extended | training | Apple | Robots.txt control token that opts a site out of Apple Intelligence training. |
OAI-SearchBot | search | OpenAI | Indexes pages for ChatGPT search results. |
Claude-SearchBot | search | Anthropic | Indexes pages so Claude can cite them. |
PerplexityBot | search | Perplexity | Indexes pages for Perplexity search results. |
ChatGPT-User | user | OpenAI | Fetches a page when a person asks ChatGPT about it. |
Claude-User | user | Anthropic | Fetches a page when a person asks Claude about it. |
Perplexity-User | user | Perplexity | Fetches a page when a person asks Perplexity about it. |