Allow Sitepager Through Your Firewall

If your site is blocking Sitepager, add a rule that allows requests whose user agent contains:

SitepagerBot

You do not need to set anything up in Sitepager first. Every request Sitepager makes to your site includes this by default.

The full user agent looks like this:

<browser user agent> SitepagerBot/1.0 (<workspace-id>; sitepager)

Match the SitepagerBot part rather than the whole string. The browser portion in front of it changes with the device a scan uses, and the workspace ID varies by workspace, so a rule that matches the whole string will miss traffic.

For the full reference, see SitepagerBot & User Agent.

The steps below depend on where your site is hosted.

---

Cloudflare

If your site uses Cloudflare (either directly or through a platform integration):

1. Go to your Cloudflare dashboard
2. Go to Security → Security rules
3. Select Create rule → Custom rules
4. Configure the rule:
   • Field: User Agent
   • Operator: contains
   • Value: SitepagerBot
   • Action: Skip
5. Under Skip, choose the Cloudflare protections that are blocking Sitepager. These may include Managed Rules, rate limiting, or Super Bot Fight Mode. Only what you choose here is skipped.
6. Save and deploy

Two Cloudflare settings this rule cannot skip:

• Bot Fight Mode on Cloudflare’s Free plan cannot be skipped with this rule. If it is blocking Sitepager, you will need to turn Bot Fight Mode off or use Super Bot Fight Mode instead.
• Under Attack Mode is controlled separately in Cloudflare. If it is blocking Sitepager, you may need to turn it off or create a separate Cloudflare rule for Sitepager.

---

Webflow

Published Webflow sites generally work with Sitepager without any special setup. Webflow does not offer its own settings for allowing Sitepager through.

Password-protected Webflow sites If your Webflow site is password-protected, use Sitepager’s login feature to enter the password before scanning. See Test Authenticated Pages.

Webflow sites that use Cloudflare If you use Cloudflare in front of your Webflow site, Cloudflare’s security settings may block Sitepager. Follow the Cloudflare steps above. Webflow documents this setup as Orange-to-Orange (O2O).

---

Framer

Framer sites generally work with Sitepager without any special setup. In some cases, Framer’s security may challenge or limit automated traffic, which can block a scan.

Password-protected Framer sites If your Framer site is password-protected, use Sitepager’s login feature. See Test Authenticated Pages.

Framer sites that use Cloudflare If your Framer site uses Cloudflare, follow the Cloudflare steps above.

---

Vercel, Netlify, or other hosting

Most hosting platforms do not block Sitepager by default. If you have turned on bot protection, such as Vercel’s Firewall or a similar service, create a rule that allows requests whose user agent contains SitepagerBot.

---

Custom server (Nginx, Apache, etc.)

Add a rule to your server configuration that allows requests when the user agent contains SitepagerBot.

Nginx example:

if ($http_user_agent ~* "SitepagerBot") {
    set $skip_bot_check 1;
}

Apache example:

SetEnvIfNoCase User-Agent "SitepagerBot" allow_bot

Adjust the names and logic to fit your existing setup.

---

If you use a custom user agent

Setting a custom user agent is optional. It changes what to match:

• If you use Browser-like (Recommended) and only change the bot identifier, SitepagerBot remains part of the user agent, so the rules above still work.
• If you use Raw and replace the complete user agent with your own value, SitepagerBot is no longer included. Match the value you supplied instead.

See Custom User Agent.

---

Password-protected staging sites

If your staging site requires a password or login, use Sitepager’s login feature before scanning. See Test Authenticated Pages.