OmniImpactBot

What it does

OmniImpactBot crawls websites that their owners have submitted to our platform, so we can analyze how AI assistants and answer engines see their content. It only crawls customer-submitted sites and the sites they choose to track — it does not mass-crawl the web.

Identifying the bot

OmniImpactBot identifies itself with the following User-Agent string:

Mozilla/5.0 (compatible; OmniImpactBot/1.0; +https://omniimpact.net/bot)

Crawl behavior

• Respects robots.txt, honoring rules for the OmniImpactBot token
• Small crawls — at most 10 pages per site crawl
• Runs when a customer requests analysis, and on a periodic refresh schedule
• Fetches robots.txt, sitemaps, and llms.txt

Whitelisting OmniImpactBot

If a host or WAF (SiteGround, Cloudflare, Wordfence, etc.) is challenging or blocking the bot, the site owner or host can allow it by user-agent — match the string above, or the substring OmniImpactBot. It is stable, we publish it here, and it covers every path our crawler uses.

Scope the rule to the bot challenge only. A user-agent string is public and spoofable, so until our signed-request authentication ships a rule matching it should never bypass more than bot protection — leave managed rules, rate limiting, and the rest of the WAF active. On Cloudflare Pro/Business that is a WAF custom rule with the Skip action (Security → WAF → Custom rules), skipping only the bot-protection components:

(http.user_agent contains "OmniImpactBot")

Wordfence is the exception: its firewall allowlist is IP-based and bypasses all rules for a listed address, so we do not recommend it. Temporarily raise Wordfence's rate-limiting thresholds (Firewall → Rate Limiting) during a crawl window instead.

We deliberately do not publish IP ranges to allowlist. Our crawl traffic egresses from our cloud hosting provider's shared address ranges — they are not reserved exclusively for us and can change if we move infrastructure, and allowlisting them would also allow unrelated traffic from the same provider. We are working toward Web Bot Auth (HTTP Message Signatures), the emerging IETF standard that lets a crawler cryptographically prove its identity on every request; we will publish our key material here when it ships.

Step-by-step instructions per provider — Cloudflare, Wordfence, Sucuri, SiteGround, GoDaddy and AWS WAF — are on omniimpact.net/crawler.

Opting out

To block OmniImpactBot entirely, add the following to your site's robots.txt:

User-agent: OmniImpactBot
Disallow: /

Customers can also simply remove their site from the platform to stop crawls.

OmniImpactJourneyBot (Agent Journeys)

Omni Impact also operates OmniImpactJourneyBot, the browser agent behind our Agent Journeys feature. It attempts real customer tasks — find a product, reach a quote form, look up a policy — only on sites their owners have explicitly authorized, and only when the owner starts or schedules a run. It never crawls the wider web.

It appends the following token to a standard desktop browser User-Agent, and sends an X-OmniImpact-Journey header on every request:

OmniImpactJourneyBot/1.0 (+https://omniimpact.net/bot)

Journey traffic runs on the same infrastructure as the crawler. The agent never bypasses CAPTCHAs or bot protection, never submits forms, and stops before payment or any other irreversible action. To stop journey runs against a site you control, contact the customer who authorized them, or email us below.

Contact

Questions about OmniImpactBot? Contact [email protected].