What comes out of Jumbo pages
Tested on 7 real Jumbo pages: 27 of 31 values exactly right, 0 wrong. When it isn't sure, it says so instead of guessing. Fields it pulls:
- category
- main category
- barcode (GTIN)
- name
- price
- product number
- unit price
Left empty on some test pages, not filled in wrong. A few more example pages with these values, and the rules for them get learned too:
- brand · 4 empty
Is there an Jumbo API for product data?
Not an open, documented one you can sign up for. What people use instead: the JSON the shop's own app or site loads (undocumented, it changes without notice, and the terms may not allow it), the structured data in the product pages themselves, or the visible text. An extraction script reads the pages you are allowed to fetch, the same way a browser does, and says when the layout changed.
How to scrape Jumbo product pages without an AI
Check the page source first. Product pages often carry name, price and stock as JSON-LD. If view source shows the price, a plain HTTP request is enough: no headless browser.
Fetch only what changed. Read the sitemap and skip every page whose lastmod hasn't moved since the last run.
Go slow. One request every second or two, a clear User-Agent, and nothing behind a login. A supermarket site is not a test server.
Learn the rules once. Every product page of one shop shares a structure. Send a few and get a rule per field, tested on all of them: price, unit price, size, category.
The traps on supermarket pages
Split prices. The euros and the cents sit in two elements (2 and 69), so a simple selector reads "2" or "269".
The old price next to the new one. A promotion shows both. The rule has to pick the price you pay, not the struck-through one.
Related products. The page lists other products with their own prices. The first price on the page is often not this product's.
Class names that change every deploy. Hashed CSS classes break hand-written selectors after each release. Rules learned from several pages lean on what stays.