Etsy Search Products

Purpose

Search Etsy for listings matching a free-form query (or full search URL, or shop URL, or listing-ID list) and return the matching items as structured JSON — listing ID, title, shop name + ID, primary + alternate image URLs, current price + original price + discount %, sale-end datetime, rating + review count, "Bestseller" / "Star Seller" / "Etsy's Pick" badges, free-shipping flag, shipping-from country, ready-to-ship-in-N-days indicator, returns-accepted flag, "X people have this in their cart" social-proof, item-type (Handmade / Vintage / Craft Supply / Digital), the canonical /listing/{id}/{slug} URL, ad/sponsored flag, plus the page-wide "X,XXX results, with Ads" fuzzy total and the active filter chips. Read-only — never clicks Add to Cart, Buy it Now, Favorite, Sign In, or any purchase-flow control.

When to Use

• A user pastes any of: an etsy.com/search?q=... URL, a keyword query ("hand-poured soy candle"), a keyword + category ("earrings in Jewelry"), a list of listing IDs, or a /shop/{name} URL.
• A user supplies a multi-facet filter combination (category + price range + color + ship-to + sort) and wants the top N results.
• Bulk monitoring of a saved-search across pages.
• Any flow that would otherwise scrape Etsy HTML and you want the canonical, anti-bot-aware playbook.

Workflow

Etsy is DataDome-protected (not Akamai, despite what some older notes claim — confirmed iter-1 by Server: DataDome + X-Datadome: protected + X-Dd-B: 259 on every blocked response). The site renders the listing grid server-side as HTML — there is no JSON-LD blob and no window.__INITIAL_STATE__ on the search page; listing data lives in data-listing-id-attributed <div class="v2-listing-card"> cards under <div data-search-results>. Therefore: drive a remote Browserbase session with --verified --proxies, navigate to the canonical search URL, then parse the rendered HTML.

A lightweight HTTP fetch (Browserbase cloud fetch API, raw curl, partner-less Open-API-v3) does not work for search — see Site-Specific Gotchas.

1. Build the URL

Start from https://www.etsy.com/search and append the filter surface as query params. All filters are URL-driven — no cookie / POST state required. Locale prefixes (/uk/, /de-en/, /ca/, /au/, ...) work identically: https://www.etsy.com/uk/search?q=....

For a listing-ID list input, skip search entirely and hit https://www.etsy.com/listing/{listingId} directly per ID.

For a full search URL input, use it verbatim — Etsy's URL params are stable and the canonical form is what you receive.

2. Create a verified + proxies session

DataDome is the wall. A bare session gets HTTP 403 + a DataDome CAPTCHA-delivery HTML stub. Verified mode (TLS-fingerprint + JS-challenge resolution) plus residential proxies (geo + IP-reputation diversity) is the minimum that produces a 200 in our testing.

SID=$(browse cloud sessions create --keep-alive --verified --proxies | jq -r .id)
export BROWSE_SESSION="$SID"

For non-US locales (/uk/, /de-en/, …) prefer a region-matched proxy: --region eu-central-1 for EU paths reduces the captcha rate.

3. Navigate and wait for the listing grid

browse open "https://www.etsy.com/search?q=hand-poured+soy+candle&order=most_recent" --remote
browse wait load --remote
browse wait selector --remote 'div[data-search-results]'   # the grid container
browse wait timeout 1500 --remote                          # let lazy thumbnails settle

If browse get title --remote returns etsy.com (single-word) or browse get html body --remote contains Please enable JS and disable any ad blocker, DataDome blocked the load. Try once more with a fresh session; if still blocked, treat as the anti-bot wall.

4. Pull the HTML and parse listing cards

HTML=$(browse get html body --remote)

Each result is a <div class="v2-listing-card" data-listing-id="..." data-shop-id="..." data-page-type="search"> inside <div data-search-results>. Iterate per card and extract:

5. Capture page-wide metadata

• Result count: the heading text matches /^([0-9,]+)\s+results?(?:,\s+with Ads)?/ near the top of <div data-search-results>. Etsy intentionally fuzzes large counts (e.g. "1,000,000+ results") — store the raw string as well as a parsed integer.
• Active filter chips: button[aria-pressed="true"] inside the filter pill bar, plus parse the URL params themselves.
• Pagination: a[rel="next"] href, or compute ?page=N+1 directly. nav[aria-label="Pagination"] exposes total pages.
• Search context: read data-primary-event-name (search_page vs category_page vs shop_search) and the category_id JSON value embedded in the inline log script — useful to confirm which surface you landed on if a redirect happened.

6. Paginate

Increment &page=N. Each page reloads the full DOM; there is no client-side infinite-scroll API to call directly. Re-use the same session (don't recreate) — DataDome rate-limits new sessions harder than repeated nav inside a warm session.

7. Release the session

browse cloud sessions update "$SID" --status REQUEST_RELEASE

Site-Specific Gotchas

• DataDome, not Akamai. Etsy's anti-bot is DataDome (Server: DataDome, X-Datadome: protected, X-Dd-B: 259 on blocked responses). Don't waste time on Akamai-specific evasion. Both --proxies-on and --proxies-off via the Fetch API return 403 + a captcha-delivery HTML stub (<p>Please enable JS and disable any ad blocker</p> + ct.captcha-delivery.com/i.js). The X-Datadome-Riskscore was 0.76 (no proxy) vs 0.12 (with proxy) on the same query — proxies lower the score but don't bypass the block at the Fetch layer; a verified JS-executing browser session is required.
• The lightweight Fetch API path is a dead end for /search. browse cloud fetch https://www.etsy.com/search?q=... returns 403 with or without --proxies. Same for /listing/{id} detail pages. The only Etsy surfaces that return 200 via Fetch in our testing are /robots.txt and /c/<category-slug> category-browse pages (no ?q=). For listing-ID lookups and search, you must use a verified browser session.
• /search?q= is explicitly disallowed in robots.txt. Specifically: Disallow: /search?*q=, Disallow: /search/?*q=, Disallow: /search/*?*q=, Disallow: /search/*/?*q=. Many filter-bearing variants are also disallowed: attr_*=, price_bucket=, ship_to=, search_type=, *?order=*, *min=*, *max=*. Use the canonical web surface; honor rate limits; do not run bulk-scrape patterns from a single session.
• No JSON-LD / no window.__INITIAL_STATE__ on search pages. Despite some scraping-tutorial claims, the Etsy search page renders listing data as plain server-rendered HTML inside <div data-search-results> with <div class="v2-listing-card"> children. There is no consolidated JSON blob to parse — extract per-card attributes from the DOM. (Etsy's internal Apollo / config blob is present but contains site chrome + translations, not listing payload.)
• Canonical URLs come with tracking junk. a.listing-link[href] is https://www.etsy.com/listing/{id}/{slug}?click_key=…&click_sum=…&ls=a&ga_order=…&ga_search_type=…&ga_view_type=…&ga_search_query=…&ref=search_grid-XXX-X-X&pro=…&frs=…&sts=…. Always strip from ? onward when emitting. The pro, frs, sts flags in the original URL are useful intermediate signals: pro=1 ⇒ sponsored placement, frs=1 ⇒ free-shipping promo, sts=1 ⇒ Star Seller. Confirm against the visible badge text — do not rely on URL flags alone.
• Ad / sponsored cards interleave with organic results. Etsy injects ~1 sponsored placement per 4 cards. Detect via text "Ad by Etsy Seller" / "Ad by Etsy" near the card title, and/or pro=1 in the href. The fuzzy "X results, with Ads" count includes them. Flag is_ad: true in the output so downstream callers can filter.
• The "X results, with Ads" count is intentionally fuzzed for large result sets. Beyond ~1,000 hits Etsy displays "1,000,000+ results" / "100,000+ results" instead of an exact number. Store both the raw display string and the best-effort parsed integer.
• Pagination caps at 250 pages. Even when the result count is in the millions, page=251 returns the page-1 results or an empty grid. The effective ceiling is ~16,000 listings per query. To go deeper, partition the query by category, price bucket, or explicit= item type and union the results.
• explicit= is a bit-field, not an enum. 1=Handmade, 2=Vintage, 4=Craft Supplies, 8=Digital downloads. Combinations sum: explicit=9 = Handmade + Digital, explicit=15 = all four. The default (no explicit=) returns Handmade + Vintage + Craft Supply mixed.
• Vintage means ≥ 20 years old. Etsy enforces this at listing time; the filter is reliable. Item-type badge wording on cards is "Vintage", not the year.
• ship_to= controls the "Ships to your country" chip and influences which sellers surface — Etsy weights inventory that the seller has tagged as shippable to that ISO-2. Without ship_to=, Etsy uses IP-geo to pick a default; use an explicit ship_to= to make results reproducible across proxies. The currency symbol on cards also follows ship_to=.
• The currency symbol is locale-dependent. A US-IP session sees $; a UK locale (/uk/search) sees £; an EU locale sees €. The .currency-value is the raw decimal in the displayed currency, not USD. Read .currency-symbol to disambiguate; do not assume USD.
• Star-Seller badge is per-shop, not per-listing. A listing's Star Seller flag reflects the shop's current status — it may flip across queries. Don't cache.
• "Bestseller" is per-listing, surfaced by Etsy when the listing is in the top-1% of its category by recent sales. Etsy may revoke it without notice.
• Etsy's Pick is editorial. Shows up as an explicit badge label.
• Listing-detail page is also DataDome-blocked from raw Fetch (verified iter-1: /listing/1234567890 returns 403 with the same DataDome stub). Use the warm browser session you opened for search to fan out to listing details when more fields are needed (description, full image gallery, variant matrix).
• Etsy Open API v3 is partner-gated. openapi.etsy.com/v3/application/listings/active is reachable and responds, but only with an approved app's keystring:shared_secret OAuth credential pair — "Invalid API key: should be in the format 'keystring:shared_secret'." is the literal response with an empty key. Do not advertise the Open API as a fallback to end-users without a credential. If you happen to have partner credentials, that path is dramatically more reliable than scraping and should be preferred — but it's out of reach for the generic agent runtime, so this skill leads with the browser path.
• CAPTCHA appears mid-session sporadically. If a navigation lands on the DataDome challenge page (body text "Please enable JS …"), don't retry the same URL in a tight loop — back off, re-create the session, and slow inter-request cadence below 1 req/s.
• Category-browse pages (/c/{slug}) are NOT DataDome-walled for static category landings without a ?q= query (verified iter-1: /c/jewelry?ref=catnav-10855 returns 200 OK, 1.03 MB HTML, 64 unique listing cards via Fetch even with --proxies). The renderer is identical (search2_neu), so this is a viable lightweight Fetch fallback for category-only browsing. Adding any of ?q=, ?min=…&max=…, or other filter params can flip the response to 403 / 404 / DataDome; treat the category-page Fetch path as best-effort for category-only requests, not for keyword queries.
• No application/ld+json Product blocks on the search page. (Listing-detail pages do carry product LD-JSON, but you'll need a browser session to reach them.)
• Embedded category_id is per-page-load. The inline log script carries "category_id":<int>" — useful when you arrive via /c/{slug} and want the numeric ID for cross-referencing. Note that the slugged URL is canonical for end-users; the numeric ID is internal.

Expected Output

{
  "query": "hand-poured soy candle",
  "url": "https://www.etsy.com/search?q=hand-poured+soy+candle&order=most_recent",
  "search_context": "search_page",
  "result_count_display": "26,000+ results, with Ads",
  "result_count_estimate": 26000,
  "result_count_is_fuzzy": true,
  "page": 1,
  "page_size": 64,
  "active_filters": {
    "q": "hand-poured soy candle",
    "order": "most_recent"
  },
  "currency": "USD",
  "listings": [
    {
      "listing_id": "1030725081",
      "shop_id": "24569729",
      "shop_name": "ExampleShop",
      "shop_location": "California, United States",
      "url": "https://www.etsy.com/listing/1030725081/oval-cut-natural-yellow-sapphire-stud",
      "title": "Oval Cut Natural Yellow Sapphire Stud Earrings 14K White Gold Diamond Earrings",
      "image_url": "https://i.etsystatic.com/24569729/c/597/474/94/137/il/ec19d2/3125518692/il_340x270.3125518692_2ga7.jpg",
      "image_urls_2x": "https://i.etsystatic.com/24569729/c/597/474/94/137/il/ec19d2/3125518692/il_680x540.3125518692_2ga7.jpg",
      "price_current_formatted": "$700.50",
      "price_current_raw": 700.50,
      "price_original_formatted": "$934.00",
      "price_original_raw": 934.00,
      "discount_pct": 25,
      "currency_symbol": "$",
      "rating_decimal": 4.91,
      "rating_count": 3442,
      "badges": ["Star Seller"],
      "is_ad": false,
      "free_shipping": true,
      "ready_to_ship_days": null,
      "shipping_from_country": "United States",
      "returns_accepted": null,
      "cart_social_proof": null,
      "sale_end_at": null,
      "item_type": "Handmade",
      "is_personalizable": false,
      "is_customizable": false,
      "is_made_to_order": false,
      "variant_indicators": []
    }
  ],
  "pagination": {
    "current_page": 1,
    "has_next_page": true,
    "next_page_url": "https://www.etsy.com/search?q=hand-poured+soy+candle&order=most_recent&page=2",
    "max_page_observed": 250
  }
}

Listing-ID-list input — output is the same shape with query: null, active_filters: {}, and listings[] populated by direct /listing/{id} fetches.

Shop-scoped input — search_context: "shop_search", plus "shop": {"name": "...", "shop_id": "..."} at the top level; listings[] shape unchanged.

Anti-bot wall encountered — emit:

{
  "success": false,
  "reason": "anti_bot_wall",
  "wall": "datadome",
  "url_attempted": "...",
  "evidence": "Server: DataDome · X-Datadome: protected · 403"
}