About our crawler
BarrierOpsBot/1.0 (+https://barrierops.com/bot)What it fetches, how to slow it, how to block it, and how to make it stop for good.
Last reviewed 8 September 2026.
The user agent, and how to confirm it is us
- User agent
BarrierOpsBot/1.0 (+https://barrierops.com/bot)- IP ranges
/bot/ips.json— a JSON array of Classless Inter-Domain Routing (CIDR) blocks, updated in place- Contact
- abuse@barrierops.com
- Check the source address against the published ranges at /bot/ips.json. The file is JSON, has no authentication, and is the same file our own operations use.
- Traffic claiming to be BarrierOpsBot from any other address is not us. Tell us at abuse@barrierops.com and we will confirm it in writing, which is sometimes what an incident report needs.
There is no reverse Domain Name System (DNS) check to offer yet. Our addresses do not resolve to a name of ours, so a reverse lookup will not name us and nothing should be read into that. The published ranges are the check to run until the records exist, and this page changes on the day they do.
What it fetches, and what it never does
It does
- Requests pages with GET, and reads the rendered page in a headless browser
- Follows anchors and sitemap entries
- Records the URL, the response status, the page structure and accessibility findings
- Runs JavaScript, because a page that renders client-side is a page a resident has to use
It never does
- Submits a form of any kind. It never clicks a button and never posts anything
- Signs in. Scanning is public pages only, and there is no code path that authenticates
- Fetches or stores anything from a members-only or password-protected area
- Attempt to bypass a bot wall, a rate limit, a CAPTCHA or an IP block
It only issues GET requests. It does not click, submit, sign in or write. Clicking is how a crawler submits a form or deletes something, so ours does not click.
The three kinds of scan
One of these three you did not ask for. It is the third row, and the last column is how to end it.
| Kind of scan | Who asked for it | How many pages | What is kept | How to stop it |
|---|---|---|---|---|
| Customer property | The organisation that owns the site, after verifying it owns it | Up to 25,000, by crawl | Pages, documents, findings, screenshots, DOM snapshots, evidence | The customer turns it off in their account |
| Free 5-page scan | A visitor who typed the address into our free scan | 5 | Nothing beyond the scan itself | robots.txt, or the address below |
| Prospecting | Nobody. You did not ask for this one. | At most 20, fixed, never crawled | Rule ids, WCAG criteria and counts. No screenshots, no page content, no DOM | The address below. It also deletes what we already hold |
Anything not in this table is forbidden, and that is written into our own build instructions rather than into a policy nobody checks. If you see us doing something that is not one of these three, that is a defect and we want to hear about it at abuse@barrierops.com.
We scan a small, fixed set of pages on government websites we have no relationship with, so that a first email can open with a finding instead of an introduction. We think that is legitimate and we know it is not obviously so, which is why the bounds are published, the data is minimal, and one message ends it permanently.
Rate and politeness
| Limit | Customer property | Free scan and prospecting |
|---|---|---|
| Concurrent requests per host | 2 by default, up to 8 only if the customer raises it | 1 |
| Minimum delay between requests | 250 ms | 2 seconds, regardless of what robots.txt permits |
| Crawl-delay in robots.txt | Honoured | Honoured, and never used to go faster than 2 seconds |
| Disallow for our user agent | Honoured. A disallowed path is skipped, never substituted | Same |
| On HTTP 429 or 503 | Exponential backoff, honouring Retry-After | Same |
| After three consecutive 5xx responses | The whole scan aborts and is marked failed | Same |
| Maximum depth from the starting address | 8 | Prospecting does not crawl at all — the page list is fixed |
| Repeat visits | On the customer’s schedule — weekly by default — or when they start a scan | One prospect scan per entity per 90 days |
| Global daily ceiling | None; it is the customer’s own site | A hard cap on prospect scans per day, set low enough that a mistake is small |
A city IT department that sees us as a denial-of-service source will block us and tell its peers. The limits above are lower than the law requires and lower than robots.txt allows, because that trade is worth making.
How to slow it down
- Crawl-delay in robots.txt. Honoured in full.
Robots.txt rule to slow BarrierOpsBot to one request every 30 seconds User-agent: BarrierOpsBot Crawl-delay: 30 - Disallow the expensive paths. A search endpoint or a calendar generates unbounded URLs; disallowing it costs you nothing and saves us both.
- Tell us. abuse@barrierops.com reaches a person, and we will set a per-host rate for your domain that survives any later robots.txt change.
How to block it entirely
User-agent: BarrierOpsBot
Disallow: /- robots.txt is re-read before every scan, so this takes effect on our next visit — within 24 hours in every case, and usually within the hour.
- If you want it to stop now rather than on the next read, write to us instead. That is immediate and it also covers email.
- Blocking us in robots.txt does not remove data we already hold. Asking us to stop does.
- A firewall rule on the user agent string works too, and we will not treat it as an attack or route around it.
Ask us to stop, and to delete what we hold
Send us the website address and we will add it to the suppression list. We do not ask why. Nobody will reply to talk you out of it.
Email the domain to abuse@barrierops.com
- Honoured immediately, permanently, and without asking why.
- It stops both sides. We will not scan the domain again and we will not email anyone at it again.
- It deletes what we already hold for that domain — scan results, findings, contact details — within 24 hours. There is no expiry and no re-consent: a suppressed domain stays suppressed unless somebody at that domain asks in writing to undo it.
- Any address on the site is enough. We match on the registered domain, so www.cityofloveland.org and cityofloveland.org are the same entry.
Adding your domain does not stop the free 5-page scan being run by a member of the public who types your address into it. That is a request from a visitor, not from us. Say so in your message if you want that blocked too, and we will block it.
Reporting a security problem
- Address
- security@barrierops.com
- Acknowledgement
- Within one business day, by a person. We fix it or we explain why we are not going to. You get the answer either way.
- Scope and safe harbour
- On the disclosure policy page.
- Bounty
- There is no bounty. We say so rather than implying one.
Testing the marketing site and the public forms is in scope. Do not test with a customer’s data and do not run a load test — the disclosure page says exactly where the line is.
If the crawler caused a problem
- abuse@barrierops.com reaches a person during support hours, 09:00–17:00 US Eastern, Monday to Friday, excluding US federal holidays.
- What to include: the source IP address, a timestamp with its timezone, and a few log lines. That is enough for us to find the run.
- We will tell you what the traffic was, which of the three kinds of scan it belonged to, and what we have done about it — in writing, if you need it for an incident record. If it caused an outage or a cost, say so. We would rather know.
Questions we are asked
Are you scanning us because somebody complained?
No. There is no complaint, no referral and no investigation. A prospecting scan means your entity appeared on a public list of US governments and nothing more.
Will you publish what you found about our site?
No. We never publish findings about a named entity — not on this site, not in an article, not in any published ranking, and not anonymised in a way that identifies you.
What do you keep about a site you have no relationship with?
Rule identifiers, the WCAG success criterion each maps to, a count, and how many of the pages checked were affected. No screenshots, no page content, no DOM. It is deleted after 12 months in any case.
Does blocking you in robots.txt stop our own scans if we become a customer?
Yes, and that surprises people. A customer’s property is scanned by the same crawler, so a Disallow blocks it too. Customers usually add an exception for their own verified property.
Do you sell or share any of this?
No. Nothing collected by the crawler is sold, shared or licensed, and no advertising network is involved anywhere on this site.
Can we get written confirmation that it was deleted?
Yes. Ask at abuse@barrierops.com, quoting the domain, and you will get a dated confirmation naming the domain and what was removed.
Removal requests and crawler complaints: abuse@barrierops.com. Security reports: security@barrierops.com.