A Shopify Catalog search tells you which products the catalog returned for a query, in what order, and what each record said at that moment. That’s useful. It isn’t a ChatGPT result. It can’t show what ChatGPT retrieved, what it recommended, which seller it showed, or whether anyone bought.
So every catalog finding needs a stated limit: name the stage your evidence reached, name the next stage it didn’t, and word the claim so it stops there. Shopify says this about its own preview tool: AI channels “often re-rank results based on their own logic”, so the preview is “a directional signal, not an exact prediction”. (Shopify Help Center.)
Below is a checklist for doing that, then our own first write-up run through it line by line.
The checklist: from Catalog result to shopper outcome
This is a proposed checklist, not a standard. Each row is a stage between “the product exists” and “someone bought it”. A Catalog snapshot reaches the first three rows. Everything below them needs different evidence.
| Stage | What a Catalog snapshot shows | Strongest claim it supports | What you need before claiming more |
|---|---|---|---|
| 1. In Shopify Catalog | Indirectly: a product that was returned was in the catalog | “This product was in Shopify Catalog on [date]” | For your own store, the product count or the eligibility reason shown in Sales channels > Agentic. Eligibility rules include a plan requirement, with an exception for the Agentic plan |
| 2. Returned for a search | Yes: the first page of results for one query, with whatever buyer context you sent | “With no buyer context, Catalog returned it at position 4 for this query on [date]” | Repeat runs, and runs with buyer context. Catalog accepts country, region, postal code, language, currency and intent, and personalizes for signed-in Shop buyers |
| 3. What the record says | Yes, field by field, as received, including fields Shopify generates | “Its description said [value and unit]; Shopify’s generated spec list said [value, or nothing]” | The merchant’s source document, to say whether the fact is true. Shopify marks generated fields Inferred and warns they may be inaccurate |
| 4. Retrieved by an assistant | No | None | That assistant’s own retrieved sources, from a dated, recorded session. ChatGPT also uses third-party data and content, earlier model responses and the shopper’s Memory |
| 5. Shown or recommended | No | None | A recorded answer from the assistant, with date, account state, location and exact prompt, repeated more than once |
| 6. Seller chosen | Partly: the first returned variant’s seller | “The first returned variant was sold by the store at position 2” | The assistant’s merchant list. OpenAI ranks merchants on availability, price, quality and maker status |
| 7. Can be bought in the assistant | The first variant’s checkout flag, in that response | “The native checkout flag was false on the first variant in this response” | The channel’s checkout status in your admin, and a real checkout attempt where permitted |
| 8. Sold | No | None | Your own sales, orders and sessions per AI channel in the Shopify admin |
Two rows deserve a second look.
Row 2 is smaller than it looks. Each Catalog response in our snapshot reported how many results matched: between 128 and 457 per search. We kept the first 10. A finding about “the top 10” is a finding about the first page of a much longer list, for a shopper with no location, at one moment.
Row 8 is the one you can actually see. Shopify’s Agentic section reports sales, orders, online store sessions and conversion for each AI channel. Shopify says sales combine referral-based sales and direct-checkout sales, and these reports aren’t yet available for headless stores. They won’t tell you which product an assistant recommended or why. They are, however, outcomes rather than proxies.
Seven questions to attach to every finding
Before a catalog finding goes into a report, a deck or a pitch, answer these in one line each:
- Which route? Shopify says ChatGPT and Copilot find Shopify products through Shopify Catalog. Google AI Mode and Gemini use the Google & YouTube channel, Meta its own channel, and any assistant can also crawl the page. (Shopify Help Center.)
- Which context? What country, currency, language, buyer and stated need did you send, if any?
- How deep? How many results did you read, out of how many matched?
- Which unit? Result occurrences, distinct products, variants or seller offers? Our snapshot’s 1,620 results are occurrences, not 1,620 products.
- Which field? Did the merchant write it, or did Shopify generate it?
- What does a blank mean? Absent is unknown. It isn’t zero, and it isn’t proof the field doesn’t exist somewhere else.
- Which stage? Which row of the checklist did the evidence reach?
And the date. A catalog record is a snapshot of a moving thing.
A worked case: our own first write-up
The first internal write-up of our 162-search snapshot, drafted on October 26, crossed several of these lines. An outside reviewer flagged the biggest ones that evening, and we corrected them before publishing. Here is each overreach, close to how the draft put it, which question it failed, and what we say now.
| What the first draft said | Which question it failed | What we say now |
|---|---|---|
| This is the catalog Shopify syndicates to ChatGPT, Gemini/AI Mode, Copilot and Meta | Route | Shopify Catalog is the route Shopify names for ChatGPT and Copilot. Google and Meta get products through their own sales channels |
| On the snowsuit search, “the top 3 don’t state a temperature rating” | Field and unit | That was wrong. Position 2’s title said “protects to -30 °C”, and position 9’s description rated the hood’s insulation to -30 °C. Our matcher didn’t read “minus 30” and “-30 °C” as the same value. The unit error, in full |
| Several searches had “median 0 reviews” | Blank | A missing rating is unknown, not zero. The product rating was absent on 440 of 1,620 results; 106 of those had a rating on the first variant and 334 had neither. No rating anywhere had a count of zero. The missing-rating audit |
| A single gap score ranked categories by how weak their shelves were | Unit and stage | The score was mostly a review-count score built on that blank-as-zero error. We retired it. Why we retired the score |
| On one niche search, “7 of 10 slots are one seller” | Stage | That counts places in one unpersonalized Catalog response, not what any assistant showed. Counting sellers properly |
| Native checkout-eligible share was 0, so agent-surface purchases still route through merchant storefronts | Stage | The native checkout flag was false on all 1,620 first returned variants in this response. That flag can’t tell you how a purchase inside an assistant is completed. What the checkout flag means |
| To be picked by agents in lip categories, a brand competes on review depth | Stage | Unsupported. The snapshot never observed a pick, and the conclusion rested on the retired score |
| A small engine can “map the agent shelf” | Route and stage | It maps Shopify Catalog search results for searches we wrote. “Agent shelf” implied we had seen what agents show |
None of these were fabrications. Each one started from a real observation and then took one step past it, from “returned” to “recommended”, or from “blank” to “zero”. That step is the thing to watch for. The full, corrected snapshot is in 162 shopper searches in Shopify Catalog.
How we checked
Two new observations on this page come from the saved responses, and one from a cross-check we ran the same evening.
How many results matched. Each saved Catalog response includes the tool’s own note of how many results matched the search, such as “showing 10 of 236”. Across all 162 searches, the totals ran from 128 to 457, with a median of 294.5. For the humidifier search, “humidifier for dry winter air large room quiet top fill”, it was 315. We read the first 10 of each.
Same request, different system. On October 26 we also sent some of the same requests to a web-grounded answer engine (Exa’s answer API, with a Canadian location), asking it to act as a shopping agent and name up to five products that met every requirement. Here is the snowsuit search, “toddler snowsuit rated for minus 30”, side by side. Brands are withheld.
| Shopify Catalog, top 10 | Web-grounded answer engine, up to 5 | |
|---|---|---|
| Brands | Brand A in 9 of 10 (positions 1 to 8 and 10, sold through three storefronts); Brand B at position 9 | Brand B (2 products) and Brand C (3 products, one through a large marketplace) |
| Overlap | — | None of the 5 was the same product as any Catalog top-10 listing. Brand C wasn’t in the Catalog top 10 at all |
| Temperature | Position 2’s title says “protects to -30 °C”. Position 9’s description rates the hood’s insulation to -30 °C | All 5 described as rated to -30 °C |
| Position 10 | Its record stated no temperature rating in the description or the generated metadata | Named that same product as rated only 0 to -20 °C, citing the brand’s web page, and excluded it |
Read that carefully. The answer engine isn’t ChatGPT, it had a location the Catalog search didn’t, and it read the open web, so the two lists differ in route, context and prompt all at once. We didn’t verify any temperature claim either system relayed. What it does show is the point of this page: two systems given the same request on the same evening returned different products, and one excluded a product for a fact that wasn’t in its catalog record. A Catalog list can’t stand in for an assistant’s answer.
What you can claim, written out
Here is the same humidifier finding written at three strengths. Only the first is supported by a Catalog snapshot.
- Supported: “On October 26, with no buyer context, 4 of the 10 descriptions Shopify Catalog returned for this search printed a decibel figure. Counting Shopify-generated metadata too, 6 of 10 records did. No title did.”
- Not supported without more evidence: “Humidifiers that state decibels rank higher for ‘quiet’ searches.” That needs repeated runs, buyer context and a comparison you designed. Even then it describes Catalog search, not an assistant.
- Not supported by any catalog data: “ChatGPT recommends humidifiers that state decibels.” That needs ChatGPT’s own answers, recorded and repeated, and even those are samples of your sessions, not your customers’.
Our supplied-snippet simulation tested whether stating facts changes which listing a model picks. It’s a separate kind of evidence, at stage 5, with its own limits: invented products, listings we supplied, API models rather than the ChatGPT app.
Limitations
- The checklist and the seven questions are a proposed method for bounding claims. Passing them doesn’t make a finding true; it keeps the claim inside the evidence.
- Every example here comes from one unpersonalized Shopify Catalog snapshot taken on October 26, 2026: 162 searches we wrote, the first 10 results each, one pass.
- The cross-check was one search on a different system, with a different location and prompt. It shows that two systems can differ, not how often they do, and it isn’t ChatGPT.
- We can’t see what ChatGPT, Copilot, Gemini or any other assistant shows a particular shopper. Nobody outside those companies can, and anyone who says otherwise should show you their recorded sessions and their context.
Your next step
Take the last catalog or “AI visibility” finding you were shown, by us or anyone else. Find the row in the checklist that its evidence actually reached, and rewrite the sentence so it stops there. Then open Sales channels > Agentic in your Shopify admin and note this month’s orders by AI channel. That’s your outcome baseline. A monitoring routine that keeps both in one log is in our AI product visibility guide.