Feed management API

Pre-launch API. The concepts described here are settled, but the API shape is not: request and response fields, parameters and defaults can still change. Build against it, and talk to your Luigi's Box contact before you put an integration into production.

Feeds are normally configured with your Luigi’s Box implementation contact. When you want to manage them yourself — a new market, a URL rotation, a scripted resync — the catalog configuration API does the same job.

Everything here needs a token for the https://api.<region>.luigisbox.ai/catalog audience, and a grant covering the operation — feeds, mappings, syncs and metadata are granted separately, and reading is separate from writing. See Server-to-server tokens.

The pieces

Resource	What it is
Feed	One URL, one object type, its format and its parsing options
Mapping	How source fields become catalog fields — see Field mapping. Shared with the Content API.
Sync	One pass over a catalog’s active feeds, started on schedule or by hand. Feeds run one after another, products last; the first failed run stops the pass, and the feeds after it get no run.
Run	One feed processed once: download, parse, send. Recorded whether it succeeds or not.

One catalog holds several feeds — products, categories, brands, articles — each pointing at a mapping.

Registering a feed

curl -X POST 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sources/feeds' \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://example.com/feeds/products.xml.gz",
    "catalog_type": "product",
    "format": "xml",
    "item_tag": "product",
    "identity_field": "product_id",
    "mapping_id": 17,
    "active": true
  }'

The response is the created feed, including its feed_id and version.

Field	Required	Purpose
`url`	Yes	Where to fetch the feed. HTTPS, no interactive login.
`catalog_type`	Yes	The object type in this file: `product`, `category`, `brand`, `article`
`format`	No	`xml`, `json`, `json_lines`, `csv`. Detected on every run when omitted.
`item_tag`	No	XML element that wraps one record, e.g. `product`. Give a namespaced element with its prefix, e.g. `g:item`. Detected when omitted: an RSS, RDF, Atom or YML feed’s own record element; otherwise the outermost repeating element in the first 1 MB, looking inward only through elements that occur once, when it repeats at least three times and makes up at least half of its parent’s children. Otherwise `item` is used. An XML name of at most 128 characters; anything else is a `422`.
`json_path`	No	Path to the record array in a JSON feed, e.g. `products`. Detected when omitted.
`csv_delimiter` / `csv_has_header`	No	CSV parsing options
`identity_field`	No	Source field holding the record’s identity
`mapping_id`	No	The mapping to apply
`group_id_field` / `group_primary_field`	No	Source fields for variant grouping
`delete_threshold`	No	Fraction of the catalog a run may remove, `0.0`–`1.0`
`active`	No	Defaults to `true`. `false` registers without scheduling.
`position`	No	Ordering among the catalog’s feeds
`transport_options`	No	Extra options for fetching, such as authentication headers

Format, item tag, JSON path and CSV delimiter you leave out are detected on every run from the first 1 MB of the feed’s content, after decompression. Detection never replaces a value you set. Whether the first row of a CSV file is a header is not detected: set csv_has_header to false for a file without one. Set them explicitly for a production feed: detection is a convenience, not a contract. Each run reports what it detected.

Reading and changing a feed

# One feed
curl 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sources/feeds/41' \
  -H 'Authorization: Bearer <token>'

# All of a catalog's feeds
curl -G 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sources/feeds' \
  --data-urlencode 'page=1' --data-urlencode 'size=20' \
  -H 'Authorization: Bearer <token>'

Updates are PATCH, and they carry the version you last read as expected_version:

curl -X PATCH 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sources/feeds/41' \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://example.com/feeds/products-v2.xml.gz",
    "expected_version": 7
  }'

If the feed changed since you read it, the version no longer matches and the update is rejected instead of overwriting the other change. Re-read the feed and reapply your change.

Omitted fields are left unchanged; a field sent as null is cleared. url, catalog_type, active and position cannot be cleared. See Partial updates.

DELETE deactivates a feed. Later syncs stop fetching it; a sync already running finishes it. Deactivating removes nothing from the catalog; the content the feed produced can still be removed later, like any record no active feed sends any more. To remove the content right away, delete the objects — see Content API.

Triggering a sync

Feeds are checked automatically. Trigger one by hand when you have just changed a mapping, or want to pull a fresh export immediately:

curl -X POST 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sync' \
  -H 'Authorization: Bearer <token>'

The response carries a sync_id that correlates the runs this trigger starts:

{ "sync_id": "lbsy_4n8w2c6t0q" }

sync_id is lbsy_ followed by ten lowercase Crockford base32 characters. Compare it with the sync_id on a run; it is not a run’s own identifier.

To see how the sync went, list the runs it started — one per active feed, up to the first run that failed — in the shape run history returns, newest first. If we had to retry the sync after an interruption on our side, the earlier attempt’s runs are listed too; each feed’s newest run is its outcome.

curl -G 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sync-runs' \
  --data-urlencode 'sync_id=lbsy_4n8w2c6t0q' \
  -H 'Authorization: Bearer <token>'

sync_id is required here; a sync_id the catalog has no runs of returns an empty page. As in a feed’s history, total counts at most 10,000 runs, so a total of 10000 means 10,000 or more, and a page past the newest 10,000 is rejected with 422. One run is also available by its own identifier, at /catalog/v1/lbc_8w3k2p/sync-runs/lbsr_7k2m9xq4vd.

This covers the whole catalog — every active feed on it.

Forcing a full resend

Between runs, Luigi’s Box remembers which records it has already seen, so an unchanged record is not re-sent. A mapping change therefore does not reshape records that did not change.

Reset the memory to force everything through again:

curl -X POST 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sources/feeds/41/fingerprints/reset' \
  -H 'Authorization: Bearer <token>'

Then sync. The next run re-sends every record in the feed.

Reset while no run of the feed is in progress — processing is false in its sync status. A run in progress records what it sends again, and the next run skips those records.

Do this after changing a mapping, after adding a field you want backfilled, or when a catalog and a feed have drifted apart. Do not do it routinely — it is a full reindex of that feed’s content.

To re-send everything for one sync without clearing that memory, send fingerprints as false in the body of /sync; the body is optional, and without it only changed records are sent:

curl -X POST 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sync' \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{"fingerprints": false}'

Checking status

One call answers whether the catalog as a whole is current:

Bash
JSON

curl 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sync-status' \
  -H 'Authorization: Bearer <token>'

{
  "status": "succeeded",
  "last_run_started_at": "2026-03-14T02:00:04Z",
  "last_checked_at": "2026-03-14T09:41:00Z",
  "active_feed_count": 4,
  "feed_counts": {
    "running": 0,
    "succeeded": 3,
    "partial": 0,
    "failed": 0,
    "skipped": 1,
    "never_run": 0
  }
}

The catalog’s status is rolled up from its feeds’ status, described under One feed:

Catalog `status`	Meaning
`running`	Some feed is running
`failed`	One of them failed, and none is running
`partial`	One of them delivered its items with some rejected, and none is running or failed
`never_run`	No feed has ever run
`skipped`	Every feed that has run reads `skipped`
`succeeded`	Every feed that has run succeeded or reads `skipped`; `feed_counts.never_run` says how many are yet to run

last_run_started_at is when the most recent of the feeds’ last_run began, and null when no feed has run. It is a start time: a run counted here may still be in flight. last_checked_at is when we last checked any of the feeds, including checks that found nothing to do.

feed_counts has one key for every feed status and counts the active feeds in each, so the counts always add up to active_feed_count.

An inactive feed is excluded throughout — from the status and from every count. A catalog whose feeds are all switched off reports never_run over zero counts.

The status is read from the catalog’s feeds alone; whether the catalog exists is not checked here. A catalog with no feed yet reports never_run over zero counts, and so does a catalog ID that does not exist, rather than 404 — check the ID when every count stays zero after you registered a feed.

curl 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sources/feeds/41/sync-status' \
  -H 'Authorization: Bearer <token>'

{
  "feed_id": 41,
  "status": "succeeded",
  "processing": false,
  "last_run": {
    "sync_run_id": "lbsr_7k2m9xq4vd",
    "sync_id": "lbsy_4n8w2c6t0q",
    "catalog_id": "lbc_8w3k2p",
    "feed_id": 41,
    "status": "succeeded",
    "trigger": "schedule",
    "triggered_by": null,
    "force": false,
    "fingerprints": true,
    "created_at": "2026-03-14T02:00:04Z",
    "started_at": "2026-03-14T02:00:04Z",
    "finished_at": "2026-03-14T02:04:31Z",
    "items_scanned": 207993,
    "items_sent": 207993,
    "items_skipped": 0,
    "items_no_identity": 0,
    "items_rejected": 0,
    "etag": "\"a4f1c9e2-32a1b\"",
    "last_modified": "Sat, 14 Mar 2026 01:58:12 GMT",
    "detection": {
      "format": "xml",
      "item_tag": "product",
      "json_path": null,
      "csv_delimiter": null,
      "confident": true
    },
    "generation": "3c1e7a52-8f0d-4b6e-9a41-d2f75e0c8b19",
    "status_reason": null,
    "error_message": null
  },
  "last_checked_at": "2026-03-14T09:41:00Z"
}

We check every active feed about once a minute, and record each check that finds nothing to do as a skipped run: the file had not changed, or the feed was already being processed. Those checks do not replace the feed’s outcome. last_run is the feed’s newest run that was not skipped, in the shape run history returns, and status is its status: a run in progress reads running until it finishes, and a failure stays failed until a later run does work. Both read skipped only when every run of the feed was skipped, and status is never_run with a null last_run when the feed has never run.

last_checked_at is when the feed’s newest run of any kind began, skipped ones included, so a feed checked a minute ago and found unchanged shows it. It is null when the feed has never run.

processing is true while a run is in progress for this feed. A last_run that still reads running while processing is false is either starting or finishing — it shows its final status within seconds — or was cut short, and is recorded as failed about an hour after it started. processing is null when we cannot tell at that moment; the rest of the answer does not depend on it.

Run history

curl -G 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sources/feeds/41/sync-runs' \
  --data-urlencode 'size=20' \
  -H 'Authorization: Bearer <token>'

The history lists every run, skipped ones included, so most of a feed’s runs are usually skipped. To list only the runs that did work, filter by status; repeat it to accept several, and total counts the runs that match:

curl -G 'https://api.eu1.luigisbox.ai/catalog/v1/lbc_8w3k2p/sources/feeds/41/sync-runs' \
  --data-urlencode 'status=succeeded' \
  --data-urlencode 'status=partial' \
  --data-urlencode 'status=failed' \
  -H 'Authorization: Bearer <token>'

Pages reach the newest 10,000 runs that match, and total counts no further: it is the exact count below 10000, and 10000 means 10,000 or more. A page beyond them is rejected with 422. Filter by status to reach outcomes older than that.

New runs are added at the front of the history about once a minute, so a run listed at the end of one page can appear again at the start of the next. Compare sync_run_id to skip the ones you have already read; a status filter that leaves out skipped also leaves out most of the new runs.

Run `status`	Meaning
`running`	In progress
`succeeded`	Completed, and every item sent was accepted
`partial`	Completed, but some items were rejected — `items_rejected` counts them and `error_message` names the most common reasons
`failed`	Stopped — read `error_message` and `status_reason`
`skipped`	Nothing was done: the file had not changed since the last run (`not_modified`), or another run of the feed was in progress (`processing`)

A run that kept no unchanged item and had every item it sent rejected is failed, with a status_reason of all_items_rejected. When unchanged items were kept, they stay in your catalog, so the run is partial. A rejected item is sent again by the next run that reads the feed file, so once you fix what rejected it, it arrives without a full re-send. If the file has not changed since, start a sync to have it read.

More status_reason values may be added later, so treat one you do not recognise as a failure that needs no change on your side until it is documented here.

These status_reason values on a failed run need a change on your side: auth_required, forbidden, not_found, http_4xx, html_challenge, tls_error, delete_threshold, unparseable_threshold, no_items_parsed, parse_error and all_items_rejected.

These clear on a later run without any change on your side: timeout, stall, rate_limited, request_timeout, the server errors http_500, http_502, http_503, http_504 and http_5xx, and the connection errors dns_error, connection_refused, connect_timeout, read_timeout and transport_error. internal_error is a failure on our side and needs no change on yours; if it repeats, contact support.

preempted appears on a run that was stopped so that a sync you triggered could take over, or while we restarted the service; neither needs a change on your side.

decompress_error means the compressed file could not be read; if it repeats, check that your server sends a complete, valid gzip file.

not_modified and processing appear on a skipped run. interrupted (the run was stopped part-way) and abandoned (the run stopped responding and was closed later) appear on a failed run that ended before it finished; neither needs a change on your side.

end_of_generation_withheld and end_of_generation_failed appear on a succeeded or partial run in a catalog with more than one feed of the same type, when that run could not remove the records that left its feed by itself: its first run, a run after a fingerprint reset or with fingerprints set to false, the first run to read a file that had not changed for more than seven days, or a run with records that had no identity. Its items were delivered, but the final clean-up those records wait for did not run — withheld because a feed of the same type, this one included, did not finish ready for it, or because a later feed in the same sync failed; failed because the clean-up request did not go through. A later run removes them; neither needs a change on your side.

A run that finds no records fails with status_reason no_items_parsed when the file’s shape contradicts the feed’s settings — for example, an item_tag that never occurs in a file that repeats another element. error_message names the mismatch. Nothing is removed from your catalog. Each scheduled run downloads the file again and fails the same way until you correct the setting. If the failed run was a manual sync of a file unchanged since the last successful run, scheduled runs skip the file until it changes; sync again once the setting is corrected.

A run that finds no records in a file that matches the feed’s settings — an empty export, or a file whose record element cannot be identified — is not an error. It succeeds with no records when there is nothing from the feed to compare with: on the feed’s first run, after the memory of seen records was reset or skipped with fingerprints=false, or when the last successful run is more than 7 days old. Otherwise it fails with delete_threshold. After such a run, check how many records it read, not only its status.

A long stretch of skipped runs whose status_reason is not_modified means your ETag or Last-Modified headers work and the unchanged file is not downloaded. The same stretch when the file has changed means those headers are wrong — see Serving your feed.

Each run carries a sync_run_id, such as lbsr_7k2m9xq4vd, that names that run alone — see Record identifiers. It is random rather than sequential, so it carries no order: the history lists runs newest first by started_at.

The counters on each run:

Counter	Watch for
`items_scanned`	A sudden drop — fewer records in the file you serve. A gap to `items_sent` + `items_skipped` + `items_no_identity` is records that could not be read as an item
`items_sent`	A sudden drop — an incomplete export
`items_skipped`	A sudden drop to zero — fingerprinting stopped recognising your records
`items_no_identity`	Anything non-zero — records with no value in `identity_field`
`items_rejected`	Anything non-zero — records your catalog rejected; `error_message` names the most common reasons

items_scanned, etag, last_modified and detection describe the file the run read. They are recorded on a succeeded or partial run and on a failed run that has item counts. A run skipped as unchanged (not_modified) carries only etag and last_modified. They are null on every other run.

etag and last_modified are the ETag and Last-Modified headers your server sent when the run checked the file for changes, verbatim — null when it sent none. A run skipped as unchanged records them by how your server said so:

Your server answered	`etag` and `last_modified` on the skipped run
`304 Not Modified`	The values the run sent in `If-None-Match` and `If-Modified-Since`, since a `304` need not repeat them
`200` with the same `ETag` as before	The `ETag` and `Last-Modified` your server sent
`200` with no `ETag` and the same `Last-Modified` as before	`etag` `null`, and the `Last-Modified` your server sent

See Serving your feed.

detection is what the run found about the file’s format at its beginning. Its keys are named like the feed’s own format, item_tag, json_path and csv_delimiter, so a difference from what you configured shows; each run detects again for any setting you left out. item_tag is the element as the feed writes it, including any namespace prefix, e.g. g:item. format is xml, csv, json, json_lines or unknown, and more may be added later. It is unknown when the beginning of the file held nothing but whitespace; a file that begins like neither XML nor JSON, such as an error page, is reported as csv, usually with confident false. A key that does not apply to that format is null, and confident is false when the detection was ambiguous or the format is unknown. json_path is also null when the array is the whole file or none was found. detection is null when the file was empty.

Errors

Status	When
`404`	A feed’s `sync-status` or `sync-runs`, when the feed does not exist in that catalog; `sync-runs/{sync_run_id}`, when the catalog has no such run
`422`	A feed’s `sync-runs` with a `status` no run can have; a `page` past the newest 10,000 runs of a feed’s or the catalog’s `sync-runs`; the catalog’s `sync-runs` without `sync_id` or with a malformed one; a malformed `sync_run_id`
`422`	`POST /sync` with a body field other than `fingerprints`
`503`	The API cannot reach what it needs at the moment, the database or the queue a sync is scheduled on; any operation can answer it. Retry, as described in Errors and rate limits

The catalog’s sync-status never answers 404; see Checking status.

Managing mappings

Mappings live under the same catalog:

GET    /catalog/v1/{catalog_id}/mappings
POST   /catalog/v1/{catalog_id}/mappings
POST   /catalog/v1/{catalog_id}/mappings/validate
GET    /catalog/v1/{catalog_id}/mappings/{mapping_id}
PATCH  /catalog/v1/{catalog_id}/mappings/{mapping_id}
DELETE /catalog/v1/{catalog_id}/mappings/{mapping_id}

Always dry-run a change through /validate before saving it: it returns the mapped sample, what each rule did to it, and whether the mapping fails on it. See Field mapping.