{
  "id": "F10",
  "url": "https://cartforagents.com/research/shopify-catalog-vs-chatgpt-results",
  "title": "Shopify Catalog vs ChatGPT results: how to report a catalog finding",
  "description": "Use a proxy-to-outcome checklist to distinguish returned Catalog facts from live assistant retrieval, recommendations, seller choice and orders.",
  "answer": "A Shopify Catalog snapshot shows which products the catalog returned for a search, in what order, and what their records said at that moment. Before you publish or act on a catalog finding, name the stage your evidence reached and word the claim so it stops there. For outcomes, use your own per-channel sales, orders and sessions in the Agentic section of the Shopify admin.",
  "take": "Catalog research is the clearest window merchants have into AI shopping today, as long as it's reported as catalog research. We hold every finding on this site to that rule.",
  "summary": "A Shopify Catalog search shows what the catalog returned and what each record said. Here is how to word a finding so it stays true, using our own first write-up as the worked example.",
  "claims": [
    {
      "text": "Shopify says its Agentic search preview displays the raw outputs of Shopify Catalog search, that agentic storefronts often re-rank results based on their own logic, and that what the preview shows might differ from what customers see in the actual AI experience, so it should be used as a directional signal, not an exact prediction.",
      "source_title": "Shopify Help Center: Managing agentic storefronts, Reviewing your products' discoverability and ranking",
      "source_url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/agentic-home",
      "source_date": "2026-10-27",
      "method": "documented",
      "limitation": "Living help page checked on this date. It describes Shopify's preview tool; it does not say how much or how any assistant re-ranks."
    },
    {
      "text": "Shopify's Agentic section reports sales, orders, online store sessions and online store conversion for each AI channel a store sells on. Shopify says the sales figure combines referral-based sales and direct-checkout sales, and that these analytics are not yet available for headless stores.",
      "source_title": "Shopify Help Center: Managing agentic storefronts, Agentic storefronts performance data",
      "source_url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/agentic-home",
      "source_date": "2026-10-27",
      "method": "documented",
      "limitation": "Aggregate store-level outcomes per channel. They do not show which product an assistant recommended, or why."
    },
    {
      "text": "Shopify says ChatGPT and Microsoft Copilot find Shopify products through Shopify Catalog, while Google AI Mode and Gemini get them through the Google & YouTube sales channel and Meta surfaces through the Facebook and Instagram by Meta channel, and that products can also be discovered through web crawling, indexing and other product feeds.",
      "source_title": "Shopify Help Center: Shopify Catalog and product discovery for agentic storefronts",
      "source_url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/products",
      "source_date": "2026-10-27",
      "method": "documented",
      "limitation": "Living help page; Shopify notes Google AI Mode and Gemini are still rolling out to eligible stores. Routes, not ranking."
    },
    {
      "text": "Shopify's Global Catalog search accepts buyer signals for relevance and localization (country, region, postal code, language, currency and intent) and can return personalized results for a signed-in Shop buyer. Shopify marks the description, options, attributes, technical specifications, top features and selling points as Inferred: generated or enriched by Shopify, possibly absent or inaccurate, and not merchant-authored source text.",
      "source_title": "Shopify: Global Catalog MCP, Personalized search; Inferred response fields",
      "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog",
      "source_date": "2026-10-27",
      "method": "documented",
      "limitation": "Interface documentation for the catalog, not a description of any assistant's final choice."
    },
    {
      "text": "To be automatically included in Shopify Catalog, a store must be on the Starter plan or higher and not in private mode, except that Agentic plan stores that have not published an online store to a paid plan can still be included through agentic storefronts; products need a title, an image, a price above 0 and an identifiable URL.",
      "source_title": "Shopify Help Center: Requirements for being included in Shopify Catalog",
      "source_url": "https://help.shopify.com/en/manual/shopify-catalog/requirements",
      "source_date": "2026-10-27",
      "method": "documented",
      "limitation": "Admission rules only; account standing is also reviewed, with no published thresholds."
    },
    {
      "text": "OpenAI says ChatGPT selects shopping results by perceived relevance to the shopper's intent, using the query and context such as Memory or Custom instructions, structured metadata from first- and third-party providers, other third-party content and earlier model responses, and that merchants are ranked on factors like availability, price, quality and maker or primary-seller status.",
      "source_title": "OpenAI Help Center: Shopping with ChatGPT Search",
      "source_url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search",
      "source_date": "2026-10-27",
      "method": "documented",
      "limitation": "Living help page checked on this date. Inputs, not weights; it does not let anyone outside OpenAI see a given shopper's results."
    },
    {
      "text": "Our October 26, 2026 snapshot of 162 searches returned 1,620 result occurrences, not 1,620 distinct products. The product-level rating was absent on 440 of them; 106 of those carried a rating on the first variant and 334 had neither, and no rating had a count of zero. The native checkout flag was false on all 1,620 first returned variants.",
      "source_title": "Cart for Agents: Receipts cited by our guides (Shopify agent-catalog snapshot, Oct 26, 2026)",
      "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides",
      "source_date": "2026-10-26",
      "method": "observed",
      "limitation": "One unpersonalized response per search. Absent means unknown, not zero. The checkout flag covers the first variant only and does not show whether anyone could buy."
    },
    {
      "text": "In the same snapshot, the snowsuit search's position 2 title said \"protects to -30 °C\" and position 9's description rated the hood's insulation to -30 °C, but our first text matcher reported the top results as lacking a temperature rating because it did not treat \"minus 30\" and \"-30 °C\" as the same value.",
      "source_title": "Cart for Agents: Receipts cited by our guides (Shopify agent-catalog snapshot, Oct 26, 2026)",
      "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides",
      "source_date": "2026-10-26",
      "method": "observed",
      "limitation": "An error in our own tool, not evidence about any assistant."
    },
    {
      "text": "Each of the 162 Catalog responses reported how many results matched; the reported totals ranged from 128 to 457 (median 294.5), and we kept only the first 10 of each.",
      "source_title": "Cart for Agents: Catalog research limits, How we checked",
      "source_url": "https://cartforagents.com/research/shopify-catalog-vs-chatgpt-results#how-we-checked",
      "source_date": "2026-10-26",
      "method": "observed",
      "limitation": "The totals are the catalog tool's own note in each response, unpersonalized and at one moment. We did not page through them."
    },
    {
      "text": "For \"toddler snowsuit rated for minus 30\", 9 of the 10 Catalog results were one brand, sold through three storefronts. The same evening, a web-grounded answer engine given the same request returned five snowsuits from two brands, none of them the same product as any Catalog top-10 listing, and one of the two brands did not appear in the Catalog top 10. It rejected the product at Catalog position 10 as rated only 0 to -20 °C, citing the brand's web page; that product's Catalog record stated no temperature rating.",
      "source_title": "Cart for Agents: Catalog research limits, How we checked",
      "source_url": "https://cartforagents.com/research/shopify-catalog-vs-chatgpt-results#how-we-checked",
      "source_date": "2026-10-26",
      "method": "observed",
      "limitation": "One search. The answer engine was Exa's answer API with a Canadian location and a shopping-agent prompt, not ChatGPT, and the Catalog search had no location, so the two outputs differ in route, context and prompt. We did not verify either system's temperature claims."
    }
  ],
  "sources": [
    {
      "title": "Shopify Help Center: Managing agentic storefronts",
      "url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/agentic-home",
      "accessed": "2026-10-27"
    },
    {
      "title": "Shopify Help Center: Shopify Catalog and product discovery for agentic storefronts",
      "url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/products",
      "accessed": "2026-10-27"
    },
    {
      "title": "Shopify Help Center: Requirements for being included in Shopify Catalog",
      "url": "https://help.shopify.com/en/manual/shopify-catalog/requirements",
      "accessed": "2026-10-27"
    },
    {
      "title": "Shopify: Global Catalog MCP",
      "url": "https://shopify.dev/docs/agents/catalog/global-catalog",
      "accessed": "2026-10-27"
    },
    {
      "title": "OpenAI Help Center: Shopping with ChatGPT Search",
      "url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search",
      "accessed": "2026-10-27"
    },
    {
      "title": "Cart for Agents: Receipts cited by our guides",
      "url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides",
      "accessed": "2026-10-27"
    }
  ],
  "limitations": [
    "The checklist is our proposed way to bound catalog findings. It is not an industry standard, and passing it does not make a finding true; it only keeps the claim inside what the evidence reached.",
    "Our worked examples come from one unpersonalized Shopify Catalog snapshot taken on October 26, 2026: 162 searches we wrote, the first 10 results each, one pass.",
    "The cross-check used one search and a different system (a web-grounded answer engine, not ChatGPT) with a different location and prompt. It shows that two systems can return different products for the same request; it does not measure how often, or what ChatGPT shows.",
    "We cannot see what ChatGPT, Copilot, Gemini or any other assistant shows a particular shopper, and we do not claim to."
  ],
  "related": [
    "https://cartforagents.com/research/agent-shopping-research",
    "https://cartforagents.com/research/ai-visibility-score-methodology",
    "https://cartforagents.com/research/missing-product-review-data",
    "https://cartforagents.com/research/ai-product-data-unit-normalization",
    "https://cartforagents.com/guides/ai-product-visibility",
    "https://cartforagents.com/guides/how-ai-shopping-agents-choose-products"
  ],
  "published": "2026-10-27",
  "updated": "2026-10-27",
  "checked": "2026-10-27",
  "kind": "research"
}