Skip to content

About

Read the public shop identity and product catalogue of any Shopify store. No API key, no account, no browser.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

Repository files navigation

shopify-store-collector

Read the public shop identity and product catalogue of any Shopify store, from the storefront's own JSON documents (/meta.json and /products.json). No API key, no account, no browser.

  • Is it Shopify? isShopify is true, false (the site answered and is not a Shopify storefront) or null (could not tell, for example when the site blocked the request).
  • Shop identity: name, *.myshopify.com domain, currency, country, published product and collection counts.
  • Catalogue: one record per product with prices, compare-at price, discount, availability, variants count, options, images, tags and dates.
  • Prices in the shop's own currency. Shopify Markets converts storefront prices to the visitor's local currency (by IP country) while /meta.json still names the shop's currency. The collector pins the shop's currency with a currency= URL parameter, so priceMin/priceMax are in shop.currency.

Install

pip install shopify-store-collector

Python 3.9 or newer. One dependency: httpx.

Not on PyPI yet. Until the first release, install from a checkout: pip install .

Command line

shopify-store [DOMAIN ...] [-f FILE] [--max-products N] [--format json|jsonl] [-o OUTPUT]
              [--concurrency N] [--timeout S] [--delay S] [--proxy URL] [--version]
shopify-store shop.example --max-products 30
shopify-store -f stores.txt --format jsonl -o catalogue.jsonl
shopify-store shop.example --max-products 0          # shop identity only
  • Domains come from arguments and/or UTF-8 files (-f stores.txt, one per line, # starts a comment; -f - reads stdin). URLs are accepted; their host is used and a leading www. is dropped.
  • --max-products (default 1000, at most 5000): products read per store. 0 reads the shop identity only.
  • --format json (default) writes one array in input order; jsonl writes one record per line, also in input order.
  • --proxy URL (or the SHOPIFY_STORE_PROXY environment variable): an optional proxy such as http://user:pass@host:port, used for every request as given. Without it, requests go out directly.
  • Exit code 0 when the run finished (also when some stores have errors), 2 for usage errors.

Python API

from shopify_store_collector import Settings, collect

records = collect(["shop.example", "https://www.other-shop.example/"], Settings(max_products=30))
for r in records:
    print(r["domain"], r["isShopify"], (r["shop"] or {}).get("currency"), r["productsRead"])
    for p in r["products"][:3]:
        print("  ", p["title"], p["priceMin"])
import asyncio
from shopify_store_collector import Settings, ShopifyStoreCollector


async def main():
    async with ShopifyStoreCollector(Settings(max_products=100, concurrency=2)) as collector:
        record = await collector.fetch_store("shop.example")
        records = await collector.collect(["a.example", "b.example"])   # input order


asyncio.run(main())

collect() cannot be called from inside a running event loop (for example in Jupyter); use await collect_async(...) there.

Settings (all optional; the library itself reads no environment variables):

Setting Default Meaning
max_products 1000 products per store, 0 to 5000 (0 = identity only)
concurrency 4 stores read at once
timeout 25.0 seconds per request
delay 0.3 seconds between two requests to the same store
proxy None proxy URL passed to httpx unchanged
user_agent shopify-store-collector/<version> HTTP User-Agent

A request that fails with a network error or answers 429/5xx is retried once, after 2 seconds.

The record

One record per input store. Every key is always present.

Field Meaning
domain the normalised domain (null for an invalid input)
finalHost the host that served /products.json after redirects; product URLs use it
isShopify true, false or null (unknown)
shop {name, myshopifyDomain, currency, country, publishedProductsCount, publishedProductsCountCapped, publishedCollectionsCount} from /meta.json, or null
catalogueTruncated true when the store has products that are not in products (limit reached or a later page failed)
productsRead number of product records
products product records (below), in the order /products.json returns them
ok true when the catalogue was read
error null or an error code (below)
fetchedAt UTC time the record was finished, ISO 8601

publishedProductsCountCapped is true when the count shows 25,001: Shopify stops counting there, so the store has at least that many products.

Product record:

Field Meaning
productId, handle, title, url identity and storefront URL
productType, vendor, tags as set by the shop (tags at most 50)
priceMin, priceMax lowest and highest variant price, in shop.currency
compareAtMax highest compare-at price that is above its own variant's price
discountPct discount of the cheapest variant against its compare-at price, in percent
available whether any variant is available
variantsCount, options variant count; option names with their values
imageUrls, imageCount up to 4 image URLs; total image count
publishedAt, createdAt, updatedAt timestamps as the shop gives them

Example (synthetic store, shortened):

{
  "domain": "shop.example",
  "finalHost": "shop.example",
  "isShopify": true,
  "shop": {"name": "Example Outdoor Co", "myshopifyDomain": "example-outdoor.myshopify.com", "currency": "EUR",
           "country": "NL", "publishedProductsCount": 3, "publishedProductsCountCapped": false,
           "publishedCollectionsCount": 7},
  "catalogueTruncated": false,
  "productsRead": 3,
  "products": [{"productId": 9000000001, "handle": "trail-jacket", "title": "Trail Jacket",
                "url": "https://shop.example/products/trail-jacket", "priceMin": 120.0, "priceMax": 135.5,
                "compareAtMax": 150.0, "discountPct": 20.0, "available": true, "variantsCount": 3, "...": "..."}],
  "ok": true,
  "error": null,
  "fetchedAt": "2026-11-14T10:00:00Z"
}

Error codes

Code Meaning
invalid_domain the input is not a domain name; no request is made
not_shopify the site answered, but neither /meta.json nor /products.json is a Shopify document
blocked the site answered 401, 403, 429 or 503
empty_catalogue /products.json is empty while /meta.json counts published products
unexpected_response /products.json answered something else (for example a 5xx after the retry)
network_error DNS, connection or timeout error after the retry
internal_error an unexpected error for this store (the batch goes on)

When /products.json is blocked but /meta.json was read, shop is still filled.

Limits

  • Only the storefront's public JSON documents are read. Headless storefronts (a custom front end on another host) and stores that disable or block these documents come out as not_shopify or blocked.
  • Without a readable /meta.json there is no shop currency to pin, so prices may be in the caller's local currency; shop is then null.
  • /products.json pages are 250 products. Very large catalogues may stop paging before the limit; check catalogueTruncated.
  • No variant-level output (SKUs, per-variant stock), no product descriptions, no sales or best-seller data.

Be polite

This library reads public pages that the stores themselves publish. Keep the concurrency and delay modest, read only what you need, and respect each site's terms of service and robots rules. You are responsible for how you use the data.

License

MIT

About

Read the public shop identity and product catalogue of any Shopify store. No API key, no account, no browser.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages