Should I block AI training crawlers in robots.txt?

Your call. Many sites allow them; many block them. Blocking via robots.txt is honoured by the well-behaved ones (OpenAI's GPTBot, Anthropic's ClaudeBot). Doesn't stop scrapers that ignore robots.txt.

How often do these UAs change?

Search bots are very stable. Browser UAs change with each browser release (every ~6 weeks for Chrome). The tool tracks current values from each vendor's published documentation.

How can I tell a real Googlebot from a fake?

Real Googlebot is announced and verifiable via reverse DNS — the IP should reverse to a *.googlebot.com host. UA alone isn't proof; any client can claim to be Googlebot.

ua · desktop · mobile · bots · cli · live

Browse common User-Agents

Copy-ready strings for browsers, mobile, bots and CLI tools. Filter by group. Catalog runs entirely in your browser.

Common User-Agents

Total

uas

Visible

all

Group

all

scope

Action

click

copy any

catalog · user-agents

summary · groups

Desktop6

Chrome 120 / Win 11, Chrome 120 / macOS, Firefox 121 / Win, Safari 17 / macOS…

Mobile4

Safari / iPhone iOS 17, Chrome / Android 14 Pixel, Chrome / Android 13 Samsung, Safari / iPad

Bots11

Googlebot, Googlebot Mobile, Bingbot, DuckDuckBot…

CLI4

curl 8, wget, Postman, Insomnia

Click any row to copy that UA string.

Quick Start

How to pick the right User-Agent

Filter by group, search by name, click to copy. Use the result in curl or your test harness.

01
Pick a group
Desktop, Mobile, Bots or CLI.
Lookups reference the IANA registries (MIME, HTTP status, TLDs) bundled at build time — no live network calls, no rate limits, no leaked queries.
filterIANAregistryofflinelive
group · queryBots · IANA
```
Googlebot, Bingbot, …
```
loaded · IANA bundlespec · ready
02
Search
Type the platform or bot name — matches name and UA string.
Builders emit canonical syntax per the relevant RFC (9110 for HTTP, 5322 for headers, 3986 for URLs) so the output passes spec-compliant parsers.
searchRFC 9110canonicalspec-compliantlive
query · optionsios · live
```
Safari / iPhone iOS 17
```
options applied · livecanonical
03
Click to copy
Any row copies its full UA string.
Validation flags common pitfalls — uppercase header names, smart quotes inside Content-Type, missing ;charset=utf-8 — before they bite production.
copyvalidationlintpitfallslive
row · looked upuas · ready
```
Copied Googlebot
```
result · livevalidated
04
Use in curl
Pass via -A or your test harness's UA setter.
Copy writes the snippet directly into your clipboard; downloads emit the right extension and MIME so curl/httpie pick them up correctly.
useclipboardMIME-correctblobcurl-ready
ok · copiedready · done
```
curl -A "…" url
```
shipped · ready to shipsuccess

Pro tips

Web-spec details that bite later

Idempotency saves retries

POST/PUT/DELETE should be safely retried. Use idempotency keys and return the same response on duplicate requests.

Match RFC 9110 status codes

Don’t invent codes. 422 for validation errors, 409 for conflicts, 429 for rate limits. Standard codes work with off-the-shelf clients.

Set <code>Cache-Control</code> deliberately

Default browser caching is conservative. Use max-age + stale-while-revalidate for hot endpoints.

Validate Open Graph in dev

OG tags break silently. Use a previewer like OG Generator + Facebook’s debugger before launch.

CORS is a server concern

The browser sends the preflight; the server decides. Wildcard origins disable credentials — be explicit.

MIME drives behaviour

Servers must send the right Content-Type. Browsers sniff but the spec says they shouldn’t — be explicit to avoid surprises.

Use Cases

When to use a known UA

Testing, SEO, crawl behaviour.

Cross-browser QA

Spot-check rendering under each engine's UA. Bundled from IANA registries at build time — no live network call, no rate limit, no leaked queries. The MIME, TLD, and status-code tables are baked in so the tool works offline once loaded.

spec-compliant

Crawl simulation

See what Googlebot or Bingbot sees. Output is canonical per the relevant RFC (9110 for HTTP, 5322 for headers, 3986 for URLs), so spec-compliant parsers accept it as-is. Cross-browser and cross-library behaviour stays consistent.

offline

Share-card preview

Use FacebookExternalHit / Twitterbot to test OG tags. Pairs with the curl-to-fetch converter for end-to-end test stubs — copy the curl from DevTools, paste, get an idiomatic fetch() body. Drop into Playwright/Vitest fixtures without manual translation.

IANA-backed

Bot allow-lists

Compare incoming requests to known crawler UAs. Catches common pitfalls (uppercase header names, smart quotes inside Content-Type, missing ;charset=utf-8) before they bite production. Validation flags rather than auto-corrects so you stay in control.

RFC-aligned

Postman runs

Match the UA your real client will send. Validation runs as you type — copy only when the output is shaped correctly. The output drops directly into nginx, Apache, or CDN configs without re-formatting or re-quoting.

production-grade

Cache invalidation

Use a known UA when bypassing edge caches. Same input plus same options always produces identical canonical output, so the tool is safe to embed in CI scripts that diff against expected configs.

curl-ready

FAQ

Frequently asked questions

Are these real strings?

Yes — sourced from recent stable releases. Versions are kept current periodically.

Can I make my own?

Use the User-Agent Generator to compose a fresh string with chosen browser/OS/version.

Why does Googlebot use Chrome?

Mobile-first crawl uses a UA mimicking Chrome on Android with the Googlebot suffix appended.

Should I trust UA in security checks?

No — UAs are trivially spoofable. Use them for routing/UX, not for authentication.

Is anything sent to a server?

No — the catalog is static and the filter runs locally.

Suggest a tool

Something missing?

Tell us what you need. We ship the top requests every two weeks.

47 in queue·8 shipped this month

Request queued. Thanks!