Bot control

A discovery API returns structured, complete catalog data — prices, stock levels, assortment — on demand, which makes it a target for scrapers. Several independent layers sit in front of the API, each catching what the ones above it miss.

The layers

Layer	Stops
Edge protection	Volumetric attacks and large botnets
Proof of work	Cheap, high-volume crawling
Connection fingerprinting	Clients that are not browsers
Behavioural checks	Sessions that do not behave like shopping
Model scoring	Whatever survives the layers above

Edge protection. API traffic is routed through a CDN that absorbs denial-of-service traffic and known botnets. It handles volumetric cases and does not stop small, low-rate scrapers.

Proof of work. A browser has to compute a short-lived token before it can call the API — about 100–200 ms of CPU on a typical consumer device. Paid once per page load it is imperceptible to a shopper; paid per request at crawling volume it is slow and expensive. It is a deterrent, not an authorization mechanism: it does not identify the caller, it prices bulk access.

Connection fingerprinting. The properties of the TLS connection identify the kind of client making it, so a script using an HTTP library is recognized as not a browser and refused. It fingerprints the technology making the connection, not the person using it, and is not user fingerprinting in the sense the ePrivacy Directive governs.

Behavioural checks. A shopping session types, sees suggestions, clicks and navigates. A session that only issues searches and never clicks anything is treated as automated, whatever its connection looks like.

Model scoring. A model trained on session behaviour, and retrained on an ongoing basis, catches what the fixed layers do not.

Each layer is individually beatable. They combine so that defeating all of them costs more than the data is worth to most scrapers.

What this means for your integration

A normal storefront passes every layer with no special handling. Four cases need attention.

Browser code solves the challenge; server code does not

The proof of work belongs to the browser flow. A server-side integration authenticates with an OAuth client ID and secret instead, and never touches a challenge.

The two are not interchangeable. A backend script that uses a publishable key and the browser flow is refused by connection fingerprinting, because it is indistinguishable from a scraper using a stolen key. If your code runs on a server, use server-to-server tokens.

Where your code runs	Flow
A shopper’s browser	Publishable key + proof of work
Your server	OAuth client credentials

Mint the token early

Solve the challenge while the page is loading, not when the shopper starts typing. Paid in advance the cost is invisible; paid on the first keystroke it is a visible delay. See A reusable token provider.

Reporting behaviour helps you here too

The behavioural layer reads a session as a whole. An integration that searches but never reports what shoppers clicked produces sessions indistinguishable from automated querying.

Reporting clicks, cart additions and purchases improves ranking and also marks your traffic as shopper traffic.

Register your origins, and keep keys per storefront

A publishable key is bound to the origins you register and to one channel. Keep one key per storefront and register every origin it is served from, staging included. A key used from an unregistered origin is refused, so a key copied from your page source is useless elsewhere.

Legitimate automation

Not every non-browser client is hostile. Monitoring, uptime checks, internal test suites, your own data exports and approved partner integrations all have reasons to call the API without a browser.

In every case, authenticate as a service with client credentials, scoped to what that integration needs. Automation that imitates a browser is the pattern the layers above are built to catch.

If a case fits neither flow, contact Luigi’s Box.

When traffic is refused

Bot control failures surface as 403. Common causes:

Symptom	Likely cause
403 on /browser-token, mentioning origin	The page’s origin is not registered for the key
403 on /browser-token from server-side code	Connection fingerprinting — use client credentials instead
403 mentioning the nonce or the address	The challenge was reused, or redeemed from a different address
Transient 403 right after a DNS or CDN change	The challenge and the token call must resolve to the same address; retry once

Report sustained 403s on legitimate traffic with the X-Request-Id of a failing response; it identifies which layer refused the request.

Rate limiting is a separate mechanism with its own headers and a 429; see Errors and rate limits.

See also

• Browser tokens — the proof-of-work flow in detail
• Server-to-server tokens — the flow for automation
• Sending events — reporting behaviour
• Errors and rate limits