Bot protection settings

A site's bot protection is a mode, what to do with declared bots no rule names, and a list of rules; how the gate uses them is in Bot protection. Reading needs bot:read, changing bot:write. What the gate decided also needs analytics:read.

curl https://api.usherstats.com/v1/sites \
  -H "Authorization: Bearer $US_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "Shop", "hostnames": ["shop.example"]}'

A new site's protection is off:

curl https://api.usherstats.com/v1/sites/$SITE_ID/bot \
  -H "Authorization: Bearer $US_TOKEN"

Settings

• mode: off, log (decide every request and record what it would have done, while serving everyone), challenge or block. Start in log.
• bots: allow (the default), challenge or deny, for a declared bot no rule names.
• rules: checked in order; the first that matches decides. Each has an action (allow, challenge or deny), an optional label, and a match naming at least one of bot, category, ai, kind, verified, asn, country and path (a prefix). Categories are search-engine, ai-search, ai-assistant, ai-training, preview, seo, monitor, archiver, tool and other. Every field named must hold. A rule applies to a verified search engine only when it names it, so a country or network rule cannot take your site out of search.

PUT replaces all three:

curl -X PUT https://api.usherstats.com/v1/sites/$SITE_ID/bot \
  -H "Authorization: Bearer $US_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode": "log", "bots": "allow", "rules": [{"action": "deny", "label": "training crawlers", "match": {"category": "ai-training"}}, {"action": "challenge", "match": {"path": "/checkout", "asn": [16276]}}]}'

A field it cannot accept is 400, with field naming it, such as rules[0].match:

curl -X PUT https://api.usherstats.com/v1/sites/$SITE_ID/bot \
  -H "Authorization: Bearer $US_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode": "log", "rules": [{"action": "deny", "match": {}}]}'

What it decided

Every request but your own team's, files and refusals included, as action and reason, by crawler category and name, with totals per action. In log mode the reason reads log: would challenge: ....

curl "https://api.usherstats.com/v1/sites/$SITE_ID/bot/decisions?days=7" \
  -H "Authorization: Bearer $US_TOKEN"