# nginx

> nginx configuration that sends bots to Jazui through an internal proxy.

Source page: https://docs.jazui.com/integrations/nginx/en

nginx uses two `map` blocks to decide whether the request is from a bot and whether it's a static file. Bot requests are rewritten to an internal location that proxies to `https://api.jazui.com/Render`; if Jazui fails, your `index.html` is served.

## Before you start

- nginx with SSL support for the proxy (the usual build).
- Your site **verified before enabling the configuration**: with nginx, an unverified site's `403` reaches the bot.
- Your token.

## Installation

This is the full file. The two `map` blocks go in the `http` block (for example in `/etc/nginx/conf.d/jazui.conf`) and the `server` replaces or merges into your site's:

`jazui.conf` (https://docs.jazui.com/example_config/nginx.txt):

```nginx
# Jazui para nginx: los bots reciben el HTML renderizado de la SPA; el resto de visitas, la SPA normal.
# Sustituye [JAZUI_TOKEN] por el token de render (Sitios web → Token del sitio) y adapta server_name y root.
# Si Jazui no responde, se pasa del límite o falla, se sirve la SPA (el bot nunca ve el error).

# Bots (bots.txt; las listas las rellena Sync-Integrations.ps1)
map $http_user_agent $jazui_bot {
    default 0;
    # BOTS:BEGIN
    "~*(googlebot|google-inspectiontool|storebot-google|adsbot-google|bingbot|bingpreview|msnbot|adidxbot|yandex|baiduspider|duckduckbot|slurp|applebot|seznambot|qwantbot|qwantify|petalbot|sogou|naver|exabot|yetibot|360spider|soso|gptbot|chatgpt|oai-searchbot|claudebot|claude-user|claude-searchbot|anthropic-ai|perplexity|google-cloudvertexbot|bytespider|ccbot|amazonbot|meta-externalagent|cohere-ai|mistralai-user|youbot|duckassistbot|yibot|qwenbot|tongyibot|erniebot|yiyanbot|doubaobot|tiktokspider|deepseekbot|pangubot|manus-user|kimi-agent|kimi-searchbot|kimi-user|kimibot|moonshotbot|hunyuan|chatglm-spider|agi agent|ai2bot|ai2bot-deepresearcheval|ai2bot-dolma|aihitbot|amazon-kendra|amazon-qbusiness|amazonbuyforme|amzn-searchbot|amzn-user|bedrockbot|novaact|andibot|claude-web|apifybot|apifywebsitecontentcrawler|atlassian-bot|awario|bigsur\.ai|brightbot|buddybot|cartai agentic commerce|terracotta|cledarabot|cloudflare-autorag|cloudflarebrowserrenderingcrawler|cohere-training-data-crawler|cragcrawler|crawlspace|datenbank crawler|diffbot|diffbot-user|anomura|echobot bot|echoboxbot|elevenlabsbot|elevenlabscrawler|exasearchbot|exauser|factset_spyderbot|firecrawlagent|geisthaus-pagefetcher|cloudvertexbot|gemini-deep-research|google-agent|google-firebase|google-notebooklm|googleagent-mariner|googleagent-urlcontext|googleother|googleother-image|googleother-video|notebooklm|iaskbot|iaskspider|imagesiftbot|img2dataset|isscyberriskcrawler|kagi-fetcher|keenablebot|klaviyoaibot|laiondownloader|linguee bot|linkupbot|aiwebindex|meta-externalfetcher|meta-webindexer|facebookbot|azureai-searchbot|mistralai-index|mistralai-training|mozilla-tabstack|nagetbot|netestate imprint crawler|icc-crawler|oai-adsbot|panscient|panscient\.com|shapbot|payhawk invoice fetching agent|phindbot|poggio-citations|poseidon research crawler|qualifiedbot|wpbot|querit-searchbot|queritbot|quillbot|quillbot\.com|reflectionbot|cotoyogi|sbintuitionsbot|scrybot|sidetrade indexer bot|tavilybot|agenttimes|thinkbot|timpibot|twinagent|velenpublicwebcrawler|wardbot|omgili|omgilibot|grok bot|grokappandroid|grokbot|addsearchbot|agentdatabot|aranet-searchbot|bixelbot|bravebot|channel3bot|friendlycrawler|henkbot|imagespider|kangaroo bot|kunatocrawler|laion-huggingface-processor|linerbot|mycentralaiscraperbot|terra cotta|wrtnbot|zanistabot|facebookexternalhit|facebot|twitterbot|linkedinbot|slackbot|whatsapp|telegrambot|discordbot|pinterest|redditbot|skypeuripreview|embedly|quora link preview|showyoubot|outbrain|developers\.google\.com\/\+\/web\/snippet|vkshare|tumblr|bitlybot|nuzzel|flipboard|snapchat|viber|mattermost|bitrix link preview|xing-contenttabreceiver|ahrefsbot|semrushbot|mj12bot|majestic-12|dotbot|rogerbot|screaming frog|sitebulb|serpstatbot|seokicks|siteimprove|lighthouse|google page speed|gtmetrix|pingdom|uptimerobot|uptimebot|w3c_validator|w3c-checklink|archive\.org_bot|paperlibot|dataprovider\.com|grapeshotcrawler|zoominfobot|feedly|newsblur|inoreader|integration-test)" 1;
    # BOTS:END
}

# Archivos que nunca se renderizan (extensions.txt)
map $uri $jazui_static {
    default 0;
    # EXTENSIONS:BEGIN
    "~*\.(js|mjs|css|less|map|json|xml|rss|txt|webmanifest|ico|png|jpg|jpeg|gif|webp|avif|svg|bmp|tif|tiff|psd|ai|woff|woff2|ttf|otf|eot|mp3|m4a|wav|ogg|mp4|m4v|webm|avi|mov|mpg|mpeg|wmv|flv|swf|pdf|doc|docx|xls|xlsx|ppt|pptx|csv|zip|gz|rar|7z|iso|torrent|exe|dmg|apk|wasm|dll|dat|blat)$" 1;
    # EXTENSIONS:END
}

server {
    listen 443 ssl;
    server_name misitio.com;
    root /var/www/misitio;

    location / {
        set $jazui $jazui_bot;
        # Esquema antiguo de Google para las SPA, con cualquier user-agent
        if ($args ~ "_escaped_fragment_") { set $jazui 1; }
        if ($jazui_static) { set $jazui 0; }
        if ($http_x_jazui) { set $jazui 0; }
        if ($request_method !~ ^(GET|HEAD)$) { set $jazui 0; }
        # El "?" final quita la query: la URL completa viaja en X-Jazui-Url
        if ($jazui = 1) { rewrite ^ /__jazui? last; }

        try_files $uri $uri/ /index.html;
    }

    location = /__jazui {
        internal;
        proxy_http_version 1.1;
        proxy_set_header Host api.jazui.com;
        proxy_set_header Authorization "Bearer [JAZUI_TOKEN]";
        proxy_set_header X-Jazui-Url $scheme://$host$request_uri;
        # Detrás de un CDN o balanceador usa su cabecera ($http_cf_connecting_ip, $http_x_forwarded_for…)
        proxy_set_header X-Jazui-Client-IP $remote_addr;
        # Las cookies del visitante no le hacen falta a Jazui: no se envían
        proxy_set_header Cookie "";
        proxy_ssl_server_name on;
        proxy_connect_timeout 5s;
        proxy_read_timeout 20s;
        # Errores de Jazui → la SPA. 404, 410, 3xx y 403 (bot bloqueado) llegan al bot tal cual.
        proxy_intercept_errors on;
        error_page 400 401 429 500 502 503 504 = @spa;
        proxy_pass https://api.jazui.com/Render;
    }

    location @spa {
        try_files /index.html =404;
    }
}
```

Replace `[JAZUI_TOKEN]` with your token, and `server_name` and `root` with your site's. If you already have a `location /` block, add the `set` and `if` lines at the top and copy the `/__jazui` and `@spa` locations. Then check and reload:

```bash
sudo nginx -t && sudo nginx -s reload
```

## What it does

- `$jazui_bot` is 1 if the user agent is on the list; `$jazui_static`, if the path is a file.
- In `location /` they're combined with `_escaped_fragment_`, the `X-Jazui` header and the method: if it must be rendered, it's rewritten to `/__jazui`.
- `/__jazui` is internal: it sends `Authorization`, `X-Jazui-Url` (the full address) and `X-Jazui-Client-IP`, with 5 s to connect and 20 s to respond.
- With `proxy_intercept_errors`, Jazui's 400, 401, 429 and 5xx codes go to `@spa`, which serves your `index.html`.

## Keeping the bot list up to date

This configuration has the bot list written inside it, between the `BOTS:BEGIN` and `BOTS:END` markers, because the server can't download it on its own. When Jazui adds new bots:

1. In the dashboard, go to **Websites** and click **Site token** on any of your sites.
2. Under **Bots in nginx, IIS and Apache**, choose your server's tab and copy the block. You'll see how many bots it has and the list date.
3. Replace everything between `BOTS:BEGIN` and `BOTS:END` in your configuration with that block and reload the server.

You can also download the list as text with `GET https://api.jazui.com/v2/bots.txt` or the ready-made blocks with `GET https://api.jazui.com/v2/bots/config`.

## <p>Request your home page as if you were a bot, with the test user agent integration-test:</p><pre>curl -I -A "integration-test" https://mysite.com/</pre><p>The response must include x-jazui: true, with x-jazui-cache: MISS the first time and HIT afterwards. Without a bot user agent (curl -I https://mysite.com/) you should get your usual website, without x-jazui. In the dashboard, the request shows up in Bot traffic and the page in Cached pages.</p>

Request your home page as if you were a bot, with the test user agent `integration-test`:

```
curl -I -A "integration-test" https://mysite.com/
```

The response must include `x-jazui: true`, with `x-jazui-cache: MISS` the first time and `HIT` afterwards. Without a bot user agent (`curl -I https://mysite.com/`) you should get your usual website, without `x-jazui`. In the dashboard, the request shows up in **Bot traffic** and the page in **Cached pages**.

## If something doesn't work

- **Behind Cloudflare or another CDN**, `$remote_addr` is the CDN's IP: use `$http_cf_connecting_ip` in `X-Jazui-Client-IP` or configure `real_ip_header`.
- **`nginx -t` fails on `map`**: the `map` blocks must be inside `http`, not `server`.
- **SSL error with the API**: keep `proxy_ssl_server_name on` and `Host api.jazui.com`.
- **Bots get 403**: the site isn't verified or you blocked that AI bot or with a rule (check the `X-Jazui-Blocked` header).