{ "url": "https://cartforagents.com/research", "title": "Research", "articles": [ { "id": "F01", "url": "https://cartforagents.com/research/agent-shopping-research", "title": "AI shopping product research: what 162 searches returned from Shopify's agent catalog", "description": "What 162 shopper searches returned from Shopify's agent catalog on October 26, 2026: 1,620 listings, which facts were present or blank, worked examples and a mistake we made.", "answer": "On October 26, 2026 we sent 162 shopper searches in 16 categories to Shopify Catalog, with no buyer context, and kept the top 10 results for each: 1,620 returned listings. The records show which facts each listing carried at that moment, such as \"21dB\" in one humidifier description and only \"runs quietly enough to leave on overnight\" in another. The gaps were concrete: 440 listings had no product-level rating, only two descriptions gave a certificate number, and no dog-boot listing gave a paw size as a number.", "take": "The information shoppers ask about often doesn't reach the catalog agents shop from. That's the gap Cart for Agents exists to close, and these 1,620 records are where we first measured it.", "summary": "We counted what came back, kept the blanks blank, and wrote down where our own tool got it wrong. Here is what one evening of Shopify's agent catalog can teach a merchant.", "claims": [ { "text": "All 162 searches returned 10 results each, for 1,620 returned listings carrying 1,612 distinct Shopify Catalog product IDs; eight IDs appeared twice.", "source_title": "Cart for Agents: Shopify Catalog snapshot, October 26, 2026 (method)", "source_url": "https://cartforagents.com/research/agent-shopping-research#method", "source_date": "2026-10-26", "method": "observed", "limitation": "Catalog IDs group offers into one product; a distinct ID is not proof of a distinct physical product, and repeated listings are not independent market draws." }, { "text": "A product-level rating was present on 1,180 of 1,620 returned listings and absent on 440; 106 of those 440 carried a rating on the first returned variant, leaving 334 with neither. No product-level or variant rating had a count of zero.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Field presence in one catalog response. It says nothing about reviews on the product page, in review apps or elsewhere on the web. An absent rating is unknown, not zero." }, { "text": "The first returned variant was flagged as not eligible for native checkout on all 1,620 returned listings.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Unpersonalized search with no country or buyer context. It does not show that no assistant can complete a purchase." }, { "text": "For the search \"humidifier for dry winter air large room quiet top fill\", 4 of 10 product descriptions stated a decibel figure (21 dB to 28 dB), and 2 more stated one only in Shopify-generated metadata; no title stated one. The listing at position 4, in its description, gave a room size (up to 400 square feet) but described noise only as \"runs quietly enough to leave on overnight\".", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Quoted from the catalog record, not measured by us. Shopify marks description and metadata as Inferred. Six of the ten listings came from one seller, so this shelf is not a sample of the category." }, { "text": "For \"toddler snowsuit rated for minus 30\", position 2's title says \"protects to -30 °C\" and position 9's description rates the hood's insulation to \"-30°C\". Our first matcher missed both because it did not treat \"minus 30\" and \"-30 °C\" as the same value.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "An error in our analysis, corrected the same day. It is not evidence about how any assistant reads temperatures. Position 9 rates the hood's insulation, not the whole garment." }, { "text": "In the same humidifier result, one description said \"up to 32 hours\" while Shopify's generated spec list in the same record said \"Up to 36 hours\"; another generated spec list gave \"Maximum Room Area: 150 m²\", a figure absent from that description, and 150 also appears in that product's model name.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "We do not know which runtime is right, or where Shopify's 150 m² came from. The model-name match is a coincidence we can see, not a proven cause." }, { "text": "For \"anti theft backpack for city travel\", two records for what appears to be the same 28-litre backpack disagreed on size: one generated spec list gave 19.7 x 11.2 x 7.5 in, while the other record's description gave \"Height: 11.2 inch Width: 7.5 inch Depth: 2.4lbs\" and its generated spec list turned that into \"Depth 2.4 in\".", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "We did not confirm the two records are the same variant, and we do not know the true dimensions." }, { "text": "Shopify marks description, options, metadata.attributes, metadata.tech_specs, metadata.top_features, metadata.unique_selling_points and variant condition as Inferred: generated or enriched by Shopify, possibly absent or inaccurate.", "source_title": "Global Catalog MCP (Shopify developer documentation)", "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog", "source_date": "2026-10-27", "method": "documented", "limitation": "Living documentation; the date is our check date, not a publication date. It describes the interface, not how any assistant weighs these fields." }, { "text": "Shopify says ChatGPT and Microsoft Copilot find products through Shopify Catalog, while Google AI Mode and Gemini receive products through the Google & YouTube sales channel and Meta through the Facebook and Instagram channel.", "source_title": "Shopify Catalog and product discovery for agentic storefronts", "source_url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/products", "source_date": "2026-10-27", "method": "documented", "limitation": "Living merchant help page; the date is our check date. Assistants can also find products by crawling the web or from merchant feeds, so a Catalog record is one route, not the whole picture." }, { "text": "OpenAI says ChatGPT picks shopping results by relevance to the query and context, including Memory and Custom instructions.", "source_title": "Shopping with ChatGPT Search (OpenAI Help Center)", "source_url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "source_date": "2026-10-27", "method": "documented", "limitation": "Living help page; date is our check date. It lists inputs, not weights, so it cannot tell us how a Catalog record turns into a recommendation." }, { "text": "In a supplied-snippet simulation, vague copy for an invented humidifier and lip gloss was almost never picked by any of six models, and stated facts won for the gloss on every model. On GPT-6.1-Sol and two Gemini models, the humidifier with stated facts was picked in every session where the shopper's brief asked for coverage and runtime (20 of 20, 19 of 19, 15 of 15), and in none of the sessions with a generic one-line ask or no shopper (0 of 40 on each model), where the models chose competitors that already stated quieter or cheaper numbers. In the snippets the models saw, only that listing stated a room size, and it was the only one under USD 90 that did; several competitors already stated a runtime.", "source_title": "Vague vs specific product facts: simulated picks by Claude, GPT and Gemini models", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Choices among listings we supplied, with invented products. The detailed brief was written to fit the invented product, so this shows facts matching stated needs, not a random shopper. About 20 sessions per model and brief. Not live ChatGPT or Gemini shopping, visibility or sales." } ], "sources": [ { "title": "Cart for Agents: Shopify Catalog snapshot, October 26, 2026 (method on this page)", "url": "https://cartforagents.com/research/agent-shopping-research#method", "accessed": "2026-10-27" }, { "title": "Global Catalog MCP (Shopify developer documentation)", "url": "https://shopify.dev/docs/agents/catalog/global-catalog", "accessed": "2026-10-27" }, { "title": "Shopify Catalog and product discovery for agentic storefronts", "url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/products", "accessed": "2026-10-27" }, { "title": "Shopping with ChatGPT Search (OpenAI Help Center)", "url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "accessed": "2026-10-27" }, { "title": "Cart for Agents: Two humidifiers say \"quiet.\" Only one gives the number.", "url": "https://cartforagents.com/research/quiet-humidifiers", "accessed": "2026-10-27" }, { "title": "Vague vs specific product facts: simulated picks by Claude, GPT and Gemini models", "url": "https://cartforagents.com/research/vague-specific-product-facts-simulation", "accessed": "2026-10-27" } ], "limitations": [ "One unpersonalized Shopify Catalog snapshot taken over about three minutes on October 26, 2026. It is not a time series, and results can change by day, country, currency and signed-in buyer.", "The 162 searches were written by us across 16 categories we chose. They are not a random sample of shopper questions, products or merchants, and the results do not estimate how common a pattern is across Shopify.", "A Shopify Catalog response is not a ChatGPT, Copilot, Gemini or Perplexity answer. We cannot see what any assistant showed a shopper, and assistants add their own sources and ranking.", "We counted 1,620 returned listings. They include repeated sellers and eight repeated product IDs, so they are not 1,620 independent products.", "Blank fields are unknown. A missing rating is not zero reviews, and a missing decibel figure is not evidence that a product is loud.", "Several fields we read are marked Inferred by Shopify, so a value in the record may not be the merchant's own words.", "Our first scoring method was flawed and is retired. No category ranking or \"gap score\" from it is published here.", "Store and brand names, and the raw catalog responses, stay private. Only counts and short quotes are published." ], "related": [ "https://cartforagents.com/research/quiet-humidifiers", "https://cartforagents.com/research/vague-specific-product-facts-simulation", "https://cartforagents.com/guides/ai-product-visibility", "https://cartforagents.com/guides/agent-commerce-optimization", "https://cartforagents.com/guides/agent-ready-product-data" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "F03", "url": "https://cartforagents.com/research/ai-product-data-unit-normalization", "title": "Product data unit normalization: how we mistook -30 °C for a missing rating", "description": "Reconstruct our minus-30 normalization failure with raw passages, expected matches and regression cases before labeling a product fact absent.", "answer": "Two of the ten listings Shopify's agent catalog returned for \"toddler snowsuit rated for minus 30\" on October 26, 2026 already stated -30 °C, but our word-matching script didn't treat \"minus 30\" and \"-30 °C\" as the same value, and our first report wrongly said the top results didn't state a temperature rating. Before you call a temperature missing, turn sign, unit and range into one comparable value, read every field, keep the scope words, and test the matcher on lookalikes such as \"minus shipping\" and \"30° nozzle\". This was our error, not a merchant omission.", "take": "Any tool that calls a fact missing should prove it looked properly; ours didn't, the first time. We publish our mistakes because a checker you can't audit isn't worth trusting.", "summary": "Our first script told us a toddler-snowsuit shelf didn't state a -30 rating. Two listings did. Here is what went wrong, and the test cases we now run before calling any product fact missing.", "claims": [ { "text": "For \"toddler snowsuit rated for minus 30\", position 2's title says \"protects to -30 °C\" and position 9's description rates the hood's insulation to \"-30°C\". Our first matcher missed both because it did not treat \"minus 30\" and \"-30 °C\" as the same value.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "An error in our analysis, corrected the same day. It is not evidence about how any assistant reads temperatures. Position 9 rates the hood's insulation, not the whole garment." }, { "text": "Position 2's description makes two further, separate temperature statements: \"Temperatures ca. 0 - -30 °C\" and, in a list of activities, \"generally being outside in temperatures around -10°C to -30°C\". The record carries no Shopify-generated metadata.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Seller statements, quoted. Three statements, not one range; we did not inspect the garment or manufacturer documentation." }, { "text": "Position 9's generated spec list says \"Insulation: Primaloft rated to -30°C\" and its generated feature list says \"Primaloft insulation rated to -30°C keeps kids safe from the elements\"; neither keeps the word hood from the description.", "source_title": "Cart for Agents: temperature matching, how we checked", "source_url": "https://cartforagents.com/research/ai-product-data-unit-normalization#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "One record. We cannot see what Shopify used to generate these lines; the whole-garment rating is unknown to us." }, { "text": "Re-running our original matching code on the saved response: it split the query into five words (toddler, snowsuit, rated, minus, 30) and credited a listing for each word its text contained. Positions 2 and 9 each scored 3 of 5. No listing was credited for \"minus\", because none of the ten wrote the word. The code had no notion of a temperature, so it credited the bare token \"30\" wherever it appeared.", "source_title": "Cart for Agents: temperature matching, how we checked", "source_url": "https://cartforagents.com/research/ai-product-data-unit-normalization#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Describes our own retired code, reproduced on October 27 from the saved response. It is not a model of any assistant." }, { "text": "In the snowsuit result, the other eight listings stated no temperature in their title, description or Shopify-generated metadata.", "source_title": "Cart for Agents: temperature matching, how we checked", "source_url": "https://cartforagents.com/research/ai-product-data-unit-normalization#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Only the fields in this catalog response. Product pages, size charts or care labels may state more." }, { "text": "Across all 1,620 results from 162 searches, our proposed normalizer found a temperature at or below -30 °C in 8 results from 5 searches: the two snowsuits, an ice cleat, three car jump starters, a self-watering planter and an ice scraper. One jump starter's \"Rated to -22°F\" equals -30 °C, and the planter's \"Temperature resistance: -22°F to 176°F (-30°C to 80°C)\" uses the same value for its material.", "source_title": "Cart for Agents: temperature matching, how we checked", "source_url": "https://cartforagents.com/research/ai-product-data-unit-normalization#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Results are occurrences, not distinct products, from searches we wrote. The normalizer is our own proposal tested on 24 cases, not a validated tool, and it needed two fixes before passing them." }, { "text": "For \"ice cleats for running shoes on icy sidewalks\", position 4's description says the product \"can be worn in temperatures as low as -41 Fahrenheit\", while its generated spec list says \"Temperature Range: Down to 41°F\" and its generated feature list \"as low as 41°F\": the minus sign is gone.", "source_title": "Cart for Agents: temperature matching, how we checked", "source_url": "https://cartforagents.com/research/ai-product-data-unit-normalization#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "One record. -41 °F is about -40.6 °C and 41 °F is 5 °C. We cannot see how Shopify generated the line or which figure the manufacturer intends." }, { "text": "Converting Fahrenheit to Celsius: -22 °F is exactly -30 °C, -10 °F is about -23.3 °C, and -30 °F is about -34.4 °C.", "source_title": "Cart for Agents: temperature matching, how we checked", "source_url": "https://cartforagents.com/research/ai-product-data-unit-normalization#how-we-checked", "source_date": "2026-10-27", "method": "inferred", "limitation": "Arithmetic, (F - 32) x 5/9. It converts the stated number; it does not make two ratings measured differently comparable." }, { "text": "Shopify marks the description and the generated metadata fields (tech_specs, top_features, unique_selling_points) as Inferred: generated or enriched by Shopify and possibly inaccurate.", "source_title": "Shopify: Global Catalog MCP, Inferred response fields", "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog", "source_date": "2026-10-27", "method": "documented", "limitation": "Living documentation; the date is our check date. It does not describe how fields are generated or how assistants weigh them." } ], "sources": [ { "title": "Cart for Agents: receipts cited by our guides", "url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "accessed": "2026-10-27" }, { "title": "Cart for Agents: temperature matching, how we checked", "url": "https://cartforagents.com/research/ai-product-data-unit-normalization#how-we-checked", "accessed": "2026-10-27" }, { "title": "Shopify: Global Catalog MCP", "url": "https://shopify.dev/docs/agents/catalog/global-catalog", "accessed": "2026-10-27" } ], "limitations": [ "The error was in our own analysis script. It is not evidence that any AI shopping assistant misreads temperatures, and it is not a merchant omission.", "One unpersonalized Shopify agent-catalog snapshot from October 26, 2026. The 1,620 results are occurrences from 162 searches we wrote, not a sample of products or merchants.", "The proposed normalizer and its 24 test cases are ours. They catch the patterns listed here, not every way people write temperatures, and the meaning step (rating, operating range, material limit, scope) still needs a person or a separate check.", "We did not inspect any garment or manufacturer document. A stated -30 °C is a seller or Shopify statement, not a verified rating, and we do not know how it was established.", "Store and brand names are withheld. Listings are named by position." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/research/quiet-humidifiers", "https://cartforagents.com/research/missing-product-review-data", "https://cartforagents.com/guides/winter-clothing-product-data", "https://cartforagents.com/guides/product-claim-provenance" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "F05", "url": "https://cartforagents.com/research/ai-shopping-seller-concentration", "title": "Seller concentration in AI shopping results: count products, variants and sellers", "description": "Ten results can be one seller: a recount of dog-boot, backpack and tent shelves in Shopify's agent catalog by product ID, variant and seller.", "answer": "Count results, product IDs, sellers (by seller ID, never by name) and actual designs separately, because one number can't stand in for another. In an unpersonalized Shopify Catalog snapshot from October 26, 2026, a dog-boot search returned 10 product IDs from 3 sellers, and 7 of them were one seller's listings for what appears to be the same rain boot in different colours; a backpack search returned one brand from 2 sellers, with 2 models listed by both; a tent search returned 8 differently named tents from one seller. A shelf that looks crowded can be one or two sellers deep.", "take": "Some AI shopping shelves are far less competitive than they look. When ten results are really three sellers, a brand with one well-documented product has room to stand out.", "summary": "Ten results can be ten different products, one boot listed seven times, or one brand sold by two stores. We recounted three Shopify Catalog shelves so you can tell which is which on yours.", "claims": [ { "text": "For \"waterproof dog boots for small dogs in deep snow\", the top 10 results carried 10 distinct product IDs from 3 seller IDs. One seller held 7 of the 10 results, all with the same title apart from a colour suffix and all priced USD 36.00 on the first returned variant: two listings offered the same 10 colours as options, and five were single-colour listings.", "source_title": "Ten Catalog results: count products, variants and sellers separately (how we checked)", "source_url": "https://cartforagents.com/research/ai-shopping-seller-concentration#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "One unpersonalized response, first returned variant only. That the seven listings are the same physical boot is our reading of titles, options and prices; the record does not say so." }, { "text": "For \"anti theft backpack for city travel\", all 10 titles named the same brand. One seller (named like the brand's official store in the record) held 8 results and a second seller held 2. The second seller's two listings carried the same model names as two of the first seller's listings (a 28 L and a 25 L model), under different product IDs and at prices USD 2.05 to USD 3.05 higher.", "source_title": "Ten Catalog results: count products, variants and sellers separately (how we checked)", "source_url": "https://cartforagents.com/research/ai-shopping-seller-concentration#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Matching by model name in the title. We did not confirm the pairs are the same variant, and our earlier check found the two 28 L records disagreed on depth." }, { "text": "For \"ultralight backpacking tent for two people under 3 pounds\", one seller held 8 of the 10 results, and each of the 8 carried a different model name, priced USD 139.00 to USD 199.00 on the first returned variant.", "source_title": "Ten Catalog results: count products, variants and sellers separately (how we checked)", "source_url": "https://cartforagents.com/research/ai-shopping-seller-concentration#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Different model names do not prove materially different tents, and we did not check weights against the \"under 3 pounds\" request." }, { "text": "Across all 162 searches, the 1,620 results came from 1,056 distinct seller IDs but only 1,054 distinct seller names: two names each belonged to two different shops. The median search returned 8 distinct sellers; in 21 searches one seller held 5 or more of the 10 results, and in 5 searches one seller held 8.", "source_title": "Ten Catalog results: count products, variants and sellers separately (how we checked)", "source_url": "https://cartforagents.com/research/ai-shopping-seller-concentration#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Searches we wrote in 16 categories we chose, top 10 only. These counts describe this snapshot, not how concentrated Shopify categories are." }, { "text": "Every one of the 1,620 results returned exactly one variant, and each result's seller is that variant's seller. 758 results listed at least one product option with more than one value, so the returned variant was one choice among several.", "source_title": "Ten Catalog results: count products, variants and sellers separately (how we checked)", "source_url": "https://cartforagents.com/research/ai-shopping-seller-concentration#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Shopify marks product options as Inferred. Other offers grouped under the same product ID are not visible in a search response, so seller counts here are first-variant sellers only." }, { "text": "In a crude title screen, 24 of 162 searches contained two or more listings from the same seller with identical titles once text after a dash was removed. The largest case was the dog-boot search, with 7.", "source_title": "Ten Catalog results: count products, variants and sellers separately (how we checked)", "source_url": "https://cartforagents.com/research/ai-shopping-seller-concentration#how-we-checked", "source_date": "2026-10-26", "method": "inferred", "limitation": "A string-matching heuristic, not a hand audit. It misses duplicates with different wording and can flag different products that share a generic title." }, { "text": "Shopify says Global Catalog search results are clustered by Universal Product ID and include offers from multiple merchants, and that an agent calls get_product to explore options and retrieve pricing and seller checkout links for the buyer's chosen variant.", "source_title": "Global Catalog MCP (Shopify developer documentation)", "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog", "source_date": "2026-10-27", "method": "documented", "limitation": "Living documentation; the date is our check date. It does not say when two sellers' listings of one model will or will not share a product ID." }, { "text": "OpenAI says that when several merchants offer a product, ChatGPT ranks them on factors like availability, price, quality and whether they are the maker or primary seller of that item.", "source_title": "Shopping with ChatGPT Search (OpenAI Help Center)", "source_url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "source_date": "2026-10-27", "method": "documented", "limitation": "Living help page; the date is our check date. Factors are listed without weights, and a Shopify Catalog result does not show which merchant ChatGPT would list first." } ], "sources": [ { "title": "Ten Catalog results: count products, variants and sellers separately (how we checked)", "url": "https://cartforagents.com/research/ai-shopping-seller-concentration#how-we-checked", "accessed": "2026-10-27" }, { "title": "Cart for Agents: 162 shopper searches in Shopify Catalog (receipts cited by our guides)", "url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "accessed": "2026-10-27" }, { "title": "Global Catalog MCP (Shopify developer documentation)", "url": "https://shopify.dev/docs/agents/catalog/global-catalog", "accessed": "2026-10-27" }, { "title": "Shopping with ChatGPT Search (OpenAI Help Center)", "url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "accessed": "2026-10-27" } ], "limitations": [ "One unpersonalized Shopify Catalog snapshot, taken on the evening of October 26, 2026, with no country, currency or signed-in buyer. A real shopper may see a different shelf.", "Top 10 results only, for 162 searches we wrote in 16 categories we chose. The three shelves were picked because they were concentrated, so they are illustrations, not typical cases.", "Each result returned one variant and one seller. Other sellers grouped under the same product ID, and other variants, were not visible in the search response.", "Whether two listings are the same physical product is our reading of titles, options and prices. The record does not confirm it.", "A Shopify Catalog response is not a ChatGPT, Copilot or Gemini answer. Concentration on this shelf says nothing about which seller an assistant showed or who made a sale.", "Store and brand names, and the raw responses, stay private. Sellers are labelled A, B and C within each search, by number of results (A = most)." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/research/ucp-catalog-native-checkout-research", "https://cartforagents.com/guides/product-variant-structured-data", "https://cartforagents.com/guides/shopify-agentic-storefronts", "https://cartforagents.com/research/missing-product-review-data" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "F09", "url": "https://cartforagents.com/research/ai-visibility-score-methodology", "title": "AI visibility score methodology: why we retired our own score", "description": "Rebuild the failure of our first gap score using missingness, normalization errors and component contribution checks rather than a replacement grade.", "answer": "Before you trust any AI visibility or \"gap\" score, compare the weight each part was given with the share of the score it actually produced, and check how it handles blank fields and units. When we recomputed our own retired score over the 162 searches in our October 26, 2026 Shopify Catalog snapshot, review counts produced 64.7% of all points, word matching 28.0%, seller variety 7.3% and result count 0%, against design weights of 25, 30, 30 and 15 out of 100; reading blank ratings as zero supplied 591 of the review points. We stopped using the score that evening and did not replace it with another grade: we now publish plain counts with named units.", "take": "We're skeptical of single-number AI visibility scores for product brands. Ours looked rigorous and was mostly a review count, which is why we now publish what came back and what was missing instead.", "summary": "Our first catalog study gave every search a 0 to 100 \"gap score\". On paper it weighed four things evenly enough. In practice it was mostly a count of reviews, padded by blank fields read as zero. This is the audit, the record of why we stopped, and the checks to run on any score before you trust it.", "claims": [ { "text": "Re-running the original scoring code unchanged on the 162 saved Shopify Catalog responses reproduced every stored score. Across all 162 searches the score awarded 3,774.4 points: 2,442.2 (64.7%) from review depth, 1,057.2 (28.0%) from word matching, 275.0 (7.3%) from seller variety and 0 (0%) from result count.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "observed", "limitation": "A recomputation of our own retired metric on one unpersonalized snapshot of searches we wrote. It describes how this score behaved, not how any other score behaves." }, { "text": "Measured by how much of the score's spread between searches each part accounted for, review depth explained 74.0%, word matching 17.3%, seller variety 8.7% and result count 0%.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "observed", "limitation": "Each part's covariance with the total, divided by the total's variance, across 162 searches. A descriptive split, not a causal model." }, { "text": "Two parts barely moved: every search returned the 10 results requested, so the result-count part scored 0 on all 162, and 131 of 162 searches returned 6 or more distinct sellers, so the seller-variety part scored 0 on those.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "observed", "limitation": "Follows from the request limit and thresholds we chose; a different request size would change it." }, { "text": "The product-level rating was absent on 440 of 1,620 results, in 121 of 162 searches, and no rating carried a count of zero. The score read every absent rating as zero reviews.", "source_title": "Cart for Agents: 162 shopper searches in Shopify Catalog (receipts cited by our guides)", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Absent means unknown in this response, not zero reviews and not a missing review program. The search count (121) is from our recount on this page." }, { "text": "Treating absent ratings as unknown instead of zero, by taking the median of the ratings present, removed 516.4 review points across 159 searches. The other 3 searches had no product rating at all and had taken the full 25 points each from blanks alone. Together that is 591.4 points, 24.2% of all review points and 15.7% of the whole score.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "observed", "limitation": "A sensitivity check, not a corrected score. It still ignores ratings carried on the first variant and any reviews outside the catalog record." }, { "text": "Even with absent ratings treated as unknown, review depth still produced 58.8% of the points on the 159 searches with a known rating, because 105 of them had a median rating count below the score's 200 threshold.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "observed", "limitation": "Shows that fixing the blank handling alone would not have rebalanced the score. The 200 threshold was our arbitrary choice." }, { "text": "Changing only how absent ratings were handled moved 13 of the 16 categories to a different position in the score's category ranking, by up to 4 places.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "observed", "limitation": "We report how many categories moved, not either ranking. Both rankings came from the retired score and are not published." }, { "text": "For \"toddler snowsuit rated for minus 30\", the score was 53.0: 10 points for seller variety, 25 for review depth and 18 for word matching. Only 4 of 10 results carried a product-level rating, while 9 of 10 carried a rating on the first variant, and the word matcher did not treat \"minus 30\" and \"-30 °C\" as the same value.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "observed", "limitation": "One worked search. The temperature-matching error is ours, not the merchants', and is not evidence about how any assistant reads temperatures." }, { "text": "For \"humidifier for dry winter air large room quiet top fill\", the first result matched 8 of the 9 search words the matcher used, including \"quiet\", although no decibel figure appeared anywhere in its record.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "observed", "limitation": "Word overlap was computed on the title, description and Shopify-generated metadata, which Shopify marks as Inferred. One result, not a rate." }, { "text": "We infer that the score measured how few reviews a shelf's records carried and how many of the shopper's words they repeated, not whether the listings answered the shopper's need, and that it was never checked against any outcome such as picks, clicks or sales.", "source_title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "source_url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "source_date": "2026-10-27", "method": "inferred", "limitation": "Our interpretation of the audit above. Alternative readings are possible, but none would make the score a validated measure." } ], "sources": [ { "title": "AI visibility scores: audit weights and retire an invalid metric (how we checked)", "url": "https://cartforagents.com/research/ai-visibility-score-methodology#how-we-checked", "accessed": "2026-10-27" }, { "title": "Cart for Agents: 162 shopper searches in Shopify Catalog (receipts cited by our guides)", "url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "accessed": "2026-10-27" }, { "title": "Cart for Agents: 162 shopper searches in Shopify Catalog (what we retired, and why)", "url": "https://cartforagents.com/research/agent-shopping-research#what-we-retired-and-why", "accessed": "2026-10-27" } ], "limitations": [ "This audits one score we built ourselves, on one unpersonalized Shopify Catalog snapshot from October 26, 2026. It does not audit any vendor's score, and its numbers do not transfer to other metrics.", "The 162 searches were written by us in 16 categories we chose. The audit shows how the score behaved on them, not how Shopify categories compare.", "The sensitivity check (absent ratings as unknown) is a diagnostic, not a corrected score. It ignores variant-level ratings and reviews held outside the catalog record.", "A Shopify Catalog response is not a ChatGPT, Copilot or Gemini answer. Neither the retired score nor anything on this page measures what an assistant showed or recommended.", "No category ranking, per-search score table or \"opportunity\" list from the retired score is published here, and none will return under another name.", "The checklist on this page is proposed. It is not an industry standard, and passing it would not make a score predict sales." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/research/missing-product-review-data", "https://cartforagents.com/research/ai-product-data-unit-normalization", "https://cartforagents.com/research/shopify-catalog-vs-chatgpt-results", "https://cartforagents.com/research/ai-shopping-seller-concentration" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "R2", "url": "https://cartforagents.com/research/article-titles-agent-reading-simulation", "title": "Which article titles AI agents cite: a simulation across Claude, GPT and Gemini", "description": "Plain titles were cited almost every time, clever ones mostly weren't: a simulation of which article titles Claude, GPT and Gemini models cite and choose to read.", "answer": "In a supplied-snippet simulation, literal title-and-description packages for three planned Cart for Agents articles were cited in 405 of 432 Claude sessions, 144 of 144 on GPT-6.1-Sol and 143 of 144 on Gemini 3.8 Flash, against 28 of 108, 5 of 36 and 15 of 36 for deliberately clever packages. Asked to pick five of our 84 article titles, the models chose AI-behaviour and research titles for their own reading and practical product-data titles for a Shopify founder: a six-title behaviour group took 27 to 61% of picks for themselves and 0 to 2% for the founder.", "take": "Say what the page is. Plain, literal titles were cited almost every time and clever ones mostly weren't, which is why every title on this site reads like a label, not a pun.", "summary": "Plain titles that said what an article contained were cited almost every time; clever ones mostly weren't. And when we asked models what to read themselves versus what to send a busy founder, they built two very different reading lists.", "claims": [ { "text": "Across three planned articles, the four literal packages (exact contents, question, checklist, exact contents with attribution) were cited in 405 of 432 Claude sessions (three models pooled), 144 of 144 on GPT-6.1-Sol and 143 of 144 on Gemini 3.8 Flash. The clever packages were cited in 28 of 108, 5 of 36 and 15 of 36.", "source_title": "Cart for Agents: Which titles agents cite (method)", "source_url": "https://cartforagents.com/research/article-titles-agent-reading-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Supplied snippets for planned articles that had not been written or indexed; one question per article and one thin shopper ask. Title and description changed together. Exploratory cells of 12 sessions per model, with no independent confirmation round." }, { "text": "For the agentic-commerce question, the clever package was cited in 1 of 36 Claude sessions, 0 of 12 on GPT-6.1-Sol and 4 of 12 on Gemini 3.8 Flash; for the AI-shopping-agent question, 0 of 36, 0 of 12 and 0 of 12. The question and checklist packages were cited in 35 of 36, 12 of 12 and 12 of 12 for the first question.", "source_title": "Cart for Agents: Which titles agents cite (method)", "source_url": "https://cartforagents.com/research/article-titles-agent-reading-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Small cells: a 12 of 12 result has a Wilson 95% interval of about 76 to 100%. No single literal format won everywhere." }, { "text": "For the negative-product-attributes question, the clever package was already cited in 27 of 36 Claude sessions and 11 of 12 on Gemini 3.8 Flash (5 of 12 on GPT-6.1-Sol), and every literal package was cited in every session on every model.", "source_title": "Cart for Agents: Which titles agents cite (method)", "source_url": "https://cartforagents.com/research/article-titles-agent-reading-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "The real competing search results for that question were a weak topical match, so this is a ceiling effect, not evidence that one headline beat strong competitors." }, { "text": "The main outcome counted the two source IDs each model declared. 89 of 540 Claude answers also cited extra sources in the prose (87 of them from Haiku), and 23 of 180 Gemini 3.8 Flash answers had prose citations that did not match the declared IDs; GPT-6.1-Sol had none. Counting any mention in the prose, or only format-compliant answers, left the clever-versus-literal gap in place.", "source_title": "Cart for Agents: Which titles agents cite (method)", "source_url": "https://cartforagents.com/research/article-titles-agent-reading-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "The format-compliant subsets have unequal model cell sizes and are a sensitivity check, not a new confirmatory result." }, { "text": "With all 84 of our article titles shown and five picks per session, a six-title AI-behaviour group fixed before the run took 120 of 450 picks when Claude models chose for their own reading and 3 of 450 when choosing for a Shopify founder (difference 26.0 points, session-bootstrap 95% interval 22.9 to 28.9). On GPT-6.1-Sol it took 72 of 150 against 0 of 150, and on Gemini 3.8 Flash 92 of 150 against 3 of 150.", "source_title": "Cart for Agents: Which titles agents cite (method)", "source_url": "https://cartforagents.com/research/article-titles-agent-reading-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Role-framed prompts (an assigned \"free time\" role versus helping a described founder). Not human demand, conscious preference or autonomous reading. Five picks in one session are not independent, which is why the interval resamples whole sessions." }, { "text": "For the founder, \"Product specifications with units: make numbers comparable\" was picked in 82 of 90 Claude sessions, 23 of 30 on GPT-6.1-Sol and 30 of 30 on Gemini 3.8 Flash, and \"Spec-sold products: build an agent-ready category data model\" in 62 of 90, 30 of 30 and 30 of 30. Neither title was picked once for the models' own reading.", "source_title": "Cart for Agents: Which titles agents cite (method)", "source_url": "https://cartforagents.com/research/article-titles-agent-reading-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Titles only, no descriptions; titles as they stood on the test date. Picks for a described founder are not that founder's reading or buying behaviour." }, { "text": "The most-picked title for Claude's own reading, the product evidence contract (60 of 90), and the retired gap-score audit (57 of 90) were outside the pre-set behaviour group. We did not move them into it after seeing results.", "source_title": "Cart for Agents: Which titles agents cite (method)", "source_url": "https://cartforagents.com/research/article-titles-agent-reading-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Shows the shift is toward research and evaluation themes generally, not only titles about AI behaviour." }, { "text": "Every reports that changing one article's title and description to state its contents moved it from 2 of 240 to 240 of 240 citations, and that in an assigned free-time reading role, agents' picks had a correlation of 0.00 with the author's popularity ranking.", "source_title": "How to Get AI Agents to Recommend Your Articles (Every)", "source_url": "https://every.to/also-true-for-humans/how-to-get-ai-agents-to-recommend-your-articles", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Author-reported supplied-snippet simulation; the date is our check date. The popularity ranking is a proxy, not measured human traffic." } ], "sources": [ { "title": "Cart for Agents: Which titles agents cite (method)", "url": "https://cartforagents.com/research/article-titles-agent-reading-simulation#method", "accessed": "2026-10-27" }, { "title": "How to Get AI Agents to Recommend Your Articles (Every)", "url": "https://every.to/also-true-for-humans/how-to-get-ai-agents-to-recommend-your-articles", "accessed": "2026-10-27" }, { "title": "Cart for Agents: Vague vs specific product facts, simulated picks by Claude, GPT and Gemini models", "url": "https://cartforagents.com/research/vague-specific-product-facts-simulation", "accessed": "2026-10-27" } ], "limitations": [ "Supplied-snippet simulation. Models chose among titles and descriptions we gave them; nothing was searched, ranked, opened or read, and the three tested articles had not been written or indexed.", "The citation test changed the title and description together, used one question per article and one thin user ask, and ran 12 sessions per model per package. It was exploratory, with no fresh confirmation round.", "The reading test assigned roles in the prompt. It measures prompted choices, not human readership, human demand, conscious interests or autonomous agent behaviour.", "Five models from three families (three Claude models, GPT-6.1-Sol and Gemini 3.8 Flash) called through APIs at low reasoning effort. Gemini 3.1 Pro was not run on these two tests. API answers do not measure ChatGPT, Gemini apps or Google AI Mode.", "The clever packages were deliberately weak test alternatives we wrote, not real competitors' titles or our current titles." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/research/vague-specific-product-facts-simulation", "https://cartforagents.com/guides/how-ai-shopping-agents-choose-products", "https://cartforagents.com/guides/agent-ready-product-data" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "F04", "url": "https://cartforagents.com/research/missing-product-review-data", "title": "Missing product review data: a blank rating is not a zero", "description": "Audit absent, null, explicit-zero and positive review states with field paths and denominator rules before calculating summaries.", "answer": "In 1,620 results that Shopify's agent catalog returned for 162 searches on October 26, 2026, the product-level rating was absent on 440 and present on 1,180, and not one rating anywhere carried a review count of zero; 106 of the 440 had a rating on the returned variant, and 334 had neither. Treat an absent rating as unknown: count absent, null, zero and positive states separately, report how many records had the field, and calculate averages and medians only over the records that did. A blank rating is not a bad product, and it's not a zero.", "take": "If your store has reviews, make sure they reach the catalog: 334 of 1,620 listings carried no rating at all. And if you analyze shelves, never turn blanks into zeros; our own first script did.", "summary": "A blank where a rating should be is not a zero. In 1,620 catalog results we found 440 blanks and not a single zero, and our own first script still turned the blanks into zeros.", "claims": [ { "text": "Across 162 searches, the catalog returned 1,620 results (1,612 distinct Catalog product IDs). The product-level rating was present on 1,180 and absent on 440. Of those 440, 106 carried a rating on the first returned variant, and 334 had neither. No product or variant rating had a count of zero.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Occurrences, not distinct products, from searches we wrote. Field presence in one catalog response only; it says nothing about reviews on the product page, in review apps or elsewhere. Absent is unknown, not zero." }, { "text": "In every one of the 440 cases, the rating key was missing from the product record; none was present with a null value. Every product record in the response carried exactly one variant, and the variant rating was missing on 451 and present on 1,169. The product rating appeared without a variant rating on 117 results.", "source_title": "Cart for Agents: missing review data, how we checked", "source_url": "https://cartforagents.com/research/missing-product-review-data#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Counts of how the response was written, not of what any store has. Because each record carried one variant, \"first variant\" here means the only variant returned, not the product's full variant list." }, { "text": "Where both ratings were present (1,063 results), the product and variant review counts differed in 1,001: the variant count was lower in 709 and higher in 292, and equal in 62.", "source_title": "Cart for Agents: missing review data, how we checked", "source_url": "https://cartforagents.com/research/missing-product-review-data#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "We do not know how either figure is compiled; the documentation we read does not say. It shows the two levels are not interchangeable, not which is correct." }, { "text": "Of the 1,180 product ratings present, 65 had a review count of 1, 142 had 2 to 9, 346 had 10 to 99 and 627 had 100 or more. All 1,180 used a 1-to-5 scale.", "source_title": "Cart for Agents: missing review data, how we checked", "source_url": "https://cartforagents.com/research/missing-product-review-data#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Distribution of present values only. It does not explain why other records lacked a rating." }, { "text": "Our first analysis script counted every absent product rating as 0 reviews. With that zero-fill, 28 of the 162 searches showed a median of 0 reviews; 25 of those 28 had at least one product rating present, and 3 had none at all.", "source_title": "Cart for Agents: missing review data, how we checked", "source_url": "https://cartforagents.com/research/missing-product-review-data#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "An error in our own retired script, reproduced on October 27 from the saved responses. It is not evidence about how any assistant treats missing ratings." }, { "text": "For \"toddler snowsuit rated for minus 30\", 4 of 10 results carried a product rating (counts 146, 116, 11 and 139), so the zero-filled median was 0 while the median of the four present counts is 127.5. Nine of the ten carried a variant rating. For \"humidifier for dry winter air large room quiet top fill\", 3 of 10 carried a product rating (310, 17 and 1); the zero-filled median was 0, the median of present counts 17, and 7 of 10 carried a variant rating.", "source_title": "Cart for Agents: missing review data, how we checked", "source_url": "https://cartforagents.com/research/missing-product-review-data#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Two searches, ten results each. A median of four or three values is fragile; the point is the denominator, not the number." }, { "text": "In the Universal Commerce Protocol schema used by the response (version 2026-08-25), rating is an optional field on both products (\"Aggregate product rating\") and variants (\"Variant rating\"). Inside a rating, value and scale_max are required, while count (\"Number of reviews contributing to the rating\") is optional with a minimum of 0.", "source_title": "Universal Commerce Protocol: Rating, Product and Variant schemas (2026-08-25)", "source_url": "https://ucp.dev/2026-08-25/schemas/shopping/types/rating.json", "source_date": "2026-10-27", "method": "documented", "limitation": "Schema documents what a valid record may contain, checked on our access date. It does not say when a platform includes a rating or where the numbers come from." }, { "text": "Shopify's Global Catalog lets an agent filter search by variant rating: it matches products with at least one variant whose rating meets a minimum value (0 to 5 scale) and a minimum number of reviews.", "source_title": "Shopify: Global Catalog MCP, search_catalog filters", "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog", "source_date": "2026-10-27", "method": "documented", "limitation": "Living documentation, checked on this date. Our searches used no rating filter, and we do not know whether or when any assistant uses it." }, { "text": "OpenAI says ChatGPT considers the shopper's query and context, and gives as an example that it may consider available options, price, reviews and ease of use.", "source_title": "OpenAI Help Center: Shopping with ChatGPT Search", "source_url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "source_date": "2026-10-27", "method": "documented", "limitation": "Living help page; the date is our check date. It names inputs, not weights or thresholds, and does not say which review source ChatGPT reads." } ], "sources": [ { "title": "Cart for Agents: receipts cited by our guides", "url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "accessed": "2026-10-27" }, { "title": "Cart for Agents: missing review data, how we checked", "url": "https://cartforagents.com/research/missing-product-review-data#how-we-checked", "accessed": "2026-10-27" }, { "title": "Universal Commerce Protocol: Rating schema (2026-08-25)", "url": "https://ucp.dev/2026-08-25/schemas/shopping/types/rating.json", "accessed": "2026-10-27" }, { "title": "Universal Commerce Protocol: Product schema (2026-08-25)", "url": "https://ucp.dev/2026-08-25/schemas/shopping/types/product.json", "accessed": "2026-10-27" }, { "title": "Universal Commerce Protocol: Variant schema (2026-08-25)", "url": "https://ucp.dev/2026-08-25/schemas/shopping/types/variant.json", "accessed": "2026-10-27" }, { "title": "Shopify: Global Catalog MCP", "url": "https://shopify.dev/docs/agents/catalog/global-catalog", "accessed": "2026-10-27" }, { "title": "OpenAI Help Center: Shopping with ChatGPT Search", "url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "accessed": "2026-10-27" } ], "limitations": [ "One unpersonalized Shopify agent-catalog snapshot, top 10 for each of 162 searches we wrote, taken October 26, 2026. The 1,620 results are occurrences, not distinct products, and not a sample of Shopify stores.", "Field presence in a catalog response is not the same as reviews existing. A product with no rating in this record may have many reviews on its page, in a review app or on other sites.", "We do not know how Shopify fills these rating fields, or why a rating was missing on any given record. The documentation we read does not say.", "A catalog response is not a ChatGPT, Copilot or Gemini answer. Nothing here measures how any assistant treats missing ratings, and nothing here shows a ranking threshold.", "Store and brand names are withheld. The raw responses stay private; only counts and short quotes are published." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/research/quiet-humidifiers", "https://cartforagents.com/research/ai-product-data-unit-normalization", "https://cartforagents.com/research/ai-visibility-score-methodology" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "F10", "url": "https://cartforagents.com/research/shopify-catalog-vs-chatgpt-results", "title": "Shopify Catalog vs ChatGPT results: how to report a catalog finding", "description": "Use a proxy-to-outcome checklist to distinguish returned Catalog facts from live assistant retrieval, recommendations, seller choice and orders.", "answer": "A Shopify Catalog snapshot shows which products the catalog returned for a search, in what order, and what their records said at that moment. Before you publish or act on a catalog finding, name the stage your evidence reached and word the claim so it stops there. For outcomes, use your own per-channel sales, orders and sessions in the Agentic section of the Shopify admin.", "take": "Catalog research is the clearest window merchants have into AI shopping today, as long as it's reported as catalog research. We hold every finding on this site to that rule.", "summary": "A Shopify Catalog search shows what the catalog returned and what each record said. Here is how to word a finding so it stays true, using our own first write-up as the worked example.", "claims": [ { "text": "Shopify says its Agentic search preview displays the raw outputs of Shopify Catalog search, that agentic storefronts often re-rank results based on their own logic, and that what the preview shows might differ from what customers see in the actual AI experience, so it should be used as a directional signal, not an exact prediction.", "source_title": "Shopify Help Center: Managing agentic storefronts, Reviewing your products' discoverability and ranking", "source_url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/agentic-home", "source_date": "2026-10-27", "method": "documented", "limitation": "Living help page checked on this date. It describes Shopify's preview tool; it does not say how much or how any assistant re-ranks." }, { "text": "Shopify's Agentic section reports sales, orders, online store sessions and online store conversion for each AI channel a store sells on. Shopify says the sales figure combines referral-based sales and direct-checkout sales, and that these analytics are not yet available for headless stores.", "source_title": "Shopify Help Center: Managing agentic storefronts, Agentic storefronts performance data", "source_url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/agentic-home", "source_date": "2026-10-27", "method": "documented", "limitation": "Aggregate store-level outcomes per channel. They do not show which product an assistant recommended, or why." }, { "text": "Shopify says ChatGPT and Microsoft Copilot find Shopify products through Shopify Catalog, while Google AI Mode and Gemini get them through the Google & YouTube sales channel and Meta surfaces through the Facebook and Instagram by Meta channel, and that products can also be discovered through web crawling, indexing and other product feeds.", "source_title": "Shopify Help Center: Shopify Catalog and product discovery for agentic storefronts", "source_url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/products", "source_date": "2026-10-27", "method": "documented", "limitation": "Living help page; Shopify notes Google AI Mode and Gemini are still rolling out to eligible stores. Routes, not ranking." }, { "text": "Shopify's Global Catalog search accepts buyer signals for relevance and localization (country, region, postal code, language, currency and intent) and can return personalized results for a signed-in Shop buyer. Shopify marks the description, options, attributes, technical specifications, top features and selling points as Inferred: generated or enriched by Shopify, possibly absent or inaccurate, and not merchant-authored source text.", "source_title": "Shopify: Global Catalog MCP, Personalized search; Inferred response fields", "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog", "source_date": "2026-10-27", "method": "documented", "limitation": "Interface documentation for the catalog, not a description of any assistant's final choice." }, { "text": "To be automatically included in Shopify Catalog, a store must be on the Starter plan or higher and not in private mode, except that Agentic plan stores that have not published an online store to a paid plan can still be included through agentic storefronts; products need a title, an image, a price above 0 and an identifiable URL.", "source_title": "Shopify Help Center: Requirements for being included in Shopify Catalog", "source_url": "https://help.shopify.com/en/manual/shopify-catalog/requirements", "source_date": "2026-10-27", "method": "documented", "limitation": "Admission rules only; account standing is also reviewed, with no published thresholds." }, { "text": "OpenAI says ChatGPT selects shopping results by perceived relevance to the shopper's intent, using the query and context such as Memory or Custom instructions, structured metadata from first- and third-party providers, other third-party content and earlier model responses, and that merchants are ranked on factors like availability, price, quality and maker or primary-seller status.", "source_title": "OpenAI Help Center: Shopping with ChatGPT Search", "source_url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "source_date": "2026-10-27", "method": "documented", "limitation": "Living help page checked on this date. Inputs, not weights; it does not let anyone outside OpenAI see a given shopper's results." }, { "text": "Our October 26, 2026 snapshot of 162 searches returned 1,620 result occurrences, not 1,620 distinct products. The product-level rating was absent on 440 of them; 106 of those carried a rating on the first variant and 334 had neither, and no rating had a count of zero. The native checkout flag was false on all 1,620 first returned variants.", "source_title": "Cart for Agents: Receipts cited by our guides (Shopify agent-catalog snapshot, Oct 26, 2026)", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "One unpersonalized response per search. Absent means unknown, not zero. The checkout flag covers the first variant only and does not show whether anyone could buy." }, { "text": "In the same snapshot, the snowsuit search's position 2 title said \"protects to -30 °C\" and position 9's description rated the hood's insulation to -30 °C, but our first text matcher reported the top results as lacking a temperature rating because it did not treat \"minus 30\" and \"-30 °C\" as the same value.", "source_title": "Cart for Agents: Receipts cited by our guides (Shopify agent-catalog snapshot, Oct 26, 2026)", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "An error in our own tool, not evidence about any assistant." }, { "text": "Each of the 162 Catalog responses reported how many results matched; the reported totals ranged from 128 to 457 (median 294.5), and we kept only the first 10 of each.", "source_title": "Cart for Agents: Catalog research limits, How we checked", "source_url": "https://cartforagents.com/research/shopify-catalog-vs-chatgpt-results#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "The totals are the catalog tool's own note in each response, unpersonalized and at one moment. We did not page through them." }, { "text": "For \"toddler snowsuit rated for minus 30\", 9 of the 10 Catalog results were one brand, sold through three storefronts. The same evening, a web-grounded answer engine given the same request returned five snowsuits from two brands, none of them the same product as any Catalog top-10 listing, and one of the two brands did not appear in the Catalog top 10. It rejected the product at Catalog position 10 as rated only 0 to -20 °C, citing the brand's web page; that product's Catalog record stated no temperature rating.", "source_title": "Cart for Agents: Catalog research limits, How we checked", "source_url": "https://cartforagents.com/research/shopify-catalog-vs-chatgpt-results#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "One search. The answer engine was Exa's answer API with a Canadian location and a shopping-agent prompt, not ChatGPT, and the Catalog search had no location, so the two outputs differ in route, context and prompt. We did not verify either system's temperature claims." } ], "sources": [ { "title": "Shopify Help Center: Managing agentic storefronts", "url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/agentic-home", "accessed": "2026-10-27" }, { "title": "Shopify Help Center: Shopify Catalog and product discovery for agentic storefronts", "url": "https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts/products", "accessed": "2026-10-27" }, { "title": "Shopify Help Center: Requirements for being included in Shopify Catalog", "url": "https://help.shopify.com/en/manual/shopify-catalog/requirements", "accessed": "2026-10-27" }, { "title": "Shopify: Global Catalog MCP", "url": "https://shopify.dev/docs/agents/catalog/global-catalog", "accessed": "2026-10-27" }, { "title": "OpenAI Help Center: Shopping with ChatGPT Search", "url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "accessed": "2026-10-27" }, { "title": "Cart for Agents: Receipts cited by our guides", "url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "accessed": "2026-10-27" } ], "limitations": [ "The checklist is our proposed way to bound catalog findings. It is not an industry standard, and passing it does not make a finding true; it only keeps the claim inside what the evidence reached.", "Our worked examples come from one unpersonalized Shopify Catalog snapshot taken on October 26, 2026: 162 searches we wrote, the first 10 results each, one pass.", "The cross-check used one search and a different system (a web-grounded answer engine, not ChatGPT) with a different location and prompt. It shows that two systems can return different products for the same request; it does not measure how often, or what ChatGPT shows.", "We cannot see what ChatGPT, Copilot, Gemini or any other assistant shows a particular shopper, and we do not claim to." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/research/ai-visibility-score-methodology", "https://cartforagents.com/research/missing-product-review-data", "https://cartforagents.com/research/ai-product-data-unit-normalization", "https://cartforagents.com/guides/ai-product-visibility", "https://cartforagents.com/guides/how-ai-shopping-agents-choose-products" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "F08", "url": "https://cartforagents.com/research/ucp-catalog-native-checkout-research", "title": "Native checkout in Shopify Catalog: all 1,620 results said no, and every one had a buy link", "description": "Recount first-variant checkout flags in the tested unpersonalized context and separate a false flag from all live purchase routes.", "answer": "In our unpersonalized Shopify Catalog snapshot of October 26, 2026, the native-checkout flag was false on the returned variant of all 1,620 results, and every one of those variants also carried a checkout link to the seller's own store. Report it as \"the returned variant, in this search context, did not support non-redirect checkout.\"", "take": "Native checkout inside the assistant was off on every record we saw, and every product could still be bought through its seller's own link. Plan for shoppers finishing on your own site, which is also what OpenAI's merchant page now describes.", "summary": "Every product in our 1,620-result Shopify Catalog snapshot said \"no\" to native checkout, and every one still carried a link to buy. Here is what that flag means, and how to write it up.", "claims": [ { "text": "The first returned variant was flagged as not eligible for native checkout on all 1,620 returned results.", "source_title": "Cart for Agents: 162 shopper searches in Shopify Catalog (receipts cited by our guides)", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Unpersonalized search with no country, currency or buyer context. It does not show that no assistant can complete a purchase." }, { "text": "All 1,620 returned variants carried a checkout URL, and on every one it was a cart link on the same host as the seller's storefront URL.", "source_title": "Native-checkout flags: interpret a Catalog snapshot without overclaiming (how we checked)", "source_url": "https://cartforagents.com/research/ucp-catalog-native-checkout-research#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "We did not open the links or attempt a purchase, so the record shows a purchase route was offered, not that it worked." }, { "text": "Each of the 1,620 results returned exactly one variant. 758 results listed at least one product option with more than one value, and 3 returned variants were marked unavailable.", "source_title": "Native-checkout flags: interpret a Catalog snapshot without overclaiming (how we checked)", "source_url": "https://cartforagents.com/research/ucp-catalog-native-checkout-research#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Shopify marks product options as Inferred. The flag for the other variants of those products was not returned and is unknown." }, { "text": "With no country or currency passed, 1,568 returned variants were priced in US dollars and 52 in 11 other currencies.", "source_title": "Native-checkout flags: interpret a Catalog snapshot without overclaiming (how we checked)", "source_url": "https://cartforagents.com/research/ucp-catalog-native-checkout-research#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Shows that an unpersonalized search still returns a market-specific offer; it does not show which market the catalog assumed." }, { "text": "All 1,620 sellers carried Shopify shop IDs and all 1,620 variants carried Shopify variant IDs, so none of the results was an external offer. None of the returned variants carried the checkout-prerequisites field (requires).", "source_title": "Native-checkout flags: interpret a Catalog snapshot without overclaiming (how we checked)", "source_url": "https://cartforagents.com/research/ucp-catalog-native-checkout-research#how-we-checked", "source_date": "2026-10-26", "method": "observed", "limitation": "Field presence in one response. A missing requires field means unknown prerequisites, not none." }, { "text": "Shopify's Global Catalog extension reference defines eligible.native_checkout as whether the variant supports native (non-redirect) checkout, with a stated default of false.", "source_title": "Global Catalog extension (Shopify developer documentation)", "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog-extension", "source_date": "2026-10-27", "method": "documented", "limitation": "Living documentation; the date is our check date. It defines the field, not what makes a variant eligible." }, { "text": "Shopify's published JSON schema for the same extension, version 2026-08-25, gives native_checkout a default of true, which disagrees with the reference page's default of false.", "source_title": "Shopify Global Catalog Extension JSON schema (2026-08-25)", "source_url": "https://shopify.dev/ucp/schemas/2026-08-25/shopify_catalog_global.json", "source_date": "2026-10-27", "method": "documented", "limitation": "A documentation inconsistency we read on our check date. It did not affect our count, because the field was explicitly present on all 1,620 results." }, { "text": "Shopify says external offers, indexed from merchants' own websites in a beta, always have native_checkout set to false and carry no checkout URL, and that purchases for them stay on the merchant's site.", "source_title": "Global Catalog extension (Shopify developer documentation)", "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog-extension", "source_date": "2026-10-27", "method": "documented", "limitation": "Living documentation, checked a day after our snapshot. Our search used the default sources, and every result we received was a Shopify offer." }, { "text": "Shopify says that to be automatically included in Shopify Catalog a store must be on the Starter plan or higher and not in private mode, except that on the Agentic plan, products can still be included through agentic storefronts before the online store is published to a paid plan.", "source_title": "Requirements for being included in Shopify Catalog (Shopify Help Center)", "source_url": "https://help.shopify.com/en/manual/shopify-catalog/requirements", "source_date": "2026-10-27", "method": "documented", "limitation": "Living page; the date is our check date. Catalog inclusion is a separate question from native checkout." }, { "text": "OpenAI says ChatGPT shopping results include links to sites where the shopper can learn more or purchase, and that for some eligible products and merchants ChatGPT may also show an Instant Checkout option.", "source_title": "Shopping with ChatGPT Search (OpenAI Help Center)", "source_url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "source_date": "2026-10-27", "method": "documented", "limitation": "Does not list eligible merchants, products or countries, and does not mention Shopify's native-checkout flag." }, { "text": "At launch, OpenAI said Instant Checkout items are not preferred in product results, but that when ranking multiple merchants selling the same product, ChatGPT considers factors including whether Instant Checkout is enabled.", "source_title": "Buy it in ChatGPT: Instant Checkout and the Agentic Commerce Protocol (OpenAI)", "source_url": "https://openai.com/index/buy-it-in-chatgpt/", "source_date": "2025-09-29", "method": "documented", "limitation": "A launch announcement from 2025. The current help page lists merchant-ranking factors without naming Instant Checkout, and no weights are given." } ], "sources": [ { "title": "Native-checkout flags: interpret a Catalog snapshot without overclaiming (how we checked)", "url": "https://cartforagents.com/research/ucp-catalog-native-checkout-research#how-we-checked", "accessed": "2026-10-27" }, { "title": "Cart for Agents: 162 shopper searches in Shopify Catalog (receipts cited by our guides)", "url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "accessed": "2026-10-27" }, { "title": "Global Catalog extension (Shopify developer documentation)", "url": "https://shopify.dev/docs/agents/catalog/global-catalog-extension", "accessed": "2026-10-27" }, { "title": "Shopify Global Catalog Extension JSON schema (2026-08-25)", "url": "https://shopify.dev/ucp/schemas/2026-08-25/shopify_catalog_global.json", "accessed": "2026-10-27" }, { "title": "Global Catalog MCP (Shopify developer documentation)", "url": "https://shopify.dev/docs/agents/catalog/global-catalog", "accessed": "2026-10-27" }, { "title": "Requirements for being included in Shopify Catalog (Shopify Help Center)", "url": "https://help.shopify.com/en/manual/shopify-catalog/requirements", "accessed": "2026-10-27" }, { "title": "Shopping with ChatGPT Search (OpenAI Help Center)", "url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "accessed": "2026-10-27" }, { "title": "Buy it in ChatGPT: Instant Checkout and the Agentic Commerce Protocol (OpenAI, 2025-09-29)", "url": "https://openai.com/index/buy-it-in-chatgpt/", "accessed": "2026-10-27" } ], "limitations": [ "One unpersonalized Shopify Catalog snapshot on the evening of October 26, 2026: no country, currency, postal code or signed-in buyer. Shopify documents both localization and personalized search, and we tested neither.", "The flag was read on the one variant each result returned. Other variants of the same product, and other sellers grouped under the same product ID, were not returned, so their flags are unknown.", "The 162 searches were written by us in 16 categories we chose. 1,620 results are not a sample of Shopify products or stores, so \"0 of 1,620\" is not an estimate of how rare native checkout is.", "We did not open any checkout link or attempt a purchase. A checkout URL in the record shows that a route was offered, not that it worked.", "Shopify does not document the flag as the eligibility switch for ChatGPT Instant Checkout, Copilot Checkout or any other assistant program. This snapshot cannot tell you whether a store takes part in any of them.", "Store names, links and raw responses stay private. Only counts and a redacted record shape are published." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/research/ai-shopping-seller-concentration", "https://cartforagents.com/guides/shopify-agentic-storefronts", "https://cartforagents.com/guides/ai-shopping-platforms", "https://cartforagents.com/research/shopify-catalog-vs-chatgpt-results" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "R1", "url": "https://cartforagents.com/research/vague-specific-product-facts-simulation", "title": "Vague vs specific product facts: simulated picks by Claude, GPT and Gemini models", "description": "Vague listings were almost never picked; factual ones won when they answered the shopper's brief. Pick counts across Claude, GPT and Gemini models, by brief and product.", "answer": "In a supplied-snippet simulation with an invented humidifier and lip gloss, six models almost never picked the vague listings (0 to 10% of held-out sessions, against 25% for random picking), and picked the gloss with facts in its description in 82 to 100%. The factual humidifier was picked in 74% of Claude sessions and about 30% on GPT-6.1-Sol and two Gemini models: every time the shopper asked for the facts it stated, and never when the ask was simply a quiet humidifier for a bedroom.", "take": "In our tests, vague copy lost on every model. Facts won when they answered what the shopper asked and the neighbouring listings didn't, and that condition is the most useful thing a merchant can take from this experiment.", "summary": "Vague copy was shut out by every model we tried. Facts got the product back into the running. Whether they won depended on the model, the shopper's brief and what the listings beside it already said.", "claims": [ { "text": "With vague copy, the invented humidifier was picked in 3 of 180 held-out Claude sessions, 0 of 60 on GPT-6.1-Sol, 6 of 60 on Gemini 3.8 Flash and 3 of 54 on Gemini 3.1 Pro. The vague lip gloss was picked in 0 sessions on every model (0 of 180 Claude, 0 of 60 on each other model). Random picking would include it in 25% of sessions.", "source_title": "Cart for Agents: vague vs specific product facts simulation (method)", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Supplied-snippet simulation with an invented brand and invented product facts; models never searched, opened pages or bought anything. Claude counts pool three models (Haiku, Sonnet and Opus 5.5). Failed calls are excluded from n, which is why some cells are below 60." }, { "text": "Lip gloss facts written into the description were picked in 168 of 180 held-out Claude sessions (93.3%, Wilson 95% CI 88.7 to 96.1%), 60 of 60 on GPT-6.1-Sol, 53 of 60 on Gemini 3.1 Pro and 49 of 60 on Gemini 3.8 Flash.", "source_title": "Cart for Agents: vague vs specific product facts simulation (method)", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "The gloss facts (clear, non-sticky, no shimmer, fragrance-free, 5 mL) are invented, and the factual version disclosed information the vague version hid. This measures disclosure of fit facts, not better wording of the same information." }, { "text": "The humidifier facts written as a spec line were picked in 134 of 180 held-out Claude sessions (74.4%, CI 67.6 to 80.3%), 20 of 60 on GPT-6.1-Sol (33.3%, CI 22.7 to 45.9%), 19 of 59 on Gemini 3.8 Flash (32.2%) and 15 of 55 on Gemini 3.1 Pro (27.3%).", "source_title": "Cart for Agents: vague vs specific product facts simulation (method)", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "The non-Claude averages hide an all-or-nothing split by shopper brief (next claim). Model averages describe these sampled models, not whole model families or the consumer apps built on them." }, { "text": "On GPT-6.1-Sol, Gemini 3.8 Flash and Gemini 3.1 Pro, the humidifier spec line was picked in every session with the detailed shopper brief (20 of 20, 19 of 19, 15 of 15) and in none of the sessions with a one-line ask or no shopper at all (0 of 40 on each model). Claude picked it in 59 of 60, 32 of 60 and 43 of 60 sessions for the same three briefs.", "source_title": "Cart for Agents: vague vs specific product facts simulation (method)", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Small cells (about 20 sessions per model and brief). The detailed brief was written so that its needs matched the invented product, and no brief deliberately required something the product could not do." }, { "text": "In the supplied humidifier shelf, four of the seven real competing listings stated a noise level in decibels: one at 21 dB, two at 26 dB and one at 28 dB, the same figure as our invented product. Only one competitor, priced above USD 90, stated a room size.", "source_title": "Cart for Agents: vague vs specific product facts simulation (method)", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "observed", "limitation": "Counts our own supplied snippets: titles, descriptions cut to 215 characters and three feature bullets. Full product pages may state more. The figures are seller statements, not measurements." }, { "text": "In the spec-line sessions with a one-line ask or no shopper, GPT-6.1-Sol and the two Gemini models included the competitor stating 21 dB in 113 of 120 sessions, while the invented 28 dB product was never picked. Claude included the 28 dB competitor priced at USD 39.99 in all 120 of those sessions and also picked the invented product in 75 of them.", "source_title": "Cart for Agents: vague vs specific product facts simulation (method)", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Counts of which listings were picked in the spec-line sessions. They show what was chosen, not why; we did not test changing the competitors or the invented product's noise figure." }, { "text": "We infer that stating facts made the invented humidifier eligible, but it won only where its facts answered something no competitor answered (room size under the shopper's budget). Where competitors stated the same or a quieter noise figure for less money, the facts did not carry it.", "source_title": "Cart for Agents: vague vs specific product facts simulation (method)", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "inferred", "limitation": "An interpretation of pick patterns, not a tested mechanism. Brand familiarity, review counts and price differences are alternative or additional explanations that this design did not separate." }, { "text": "Adding a \"not for\" limit (with the facts in a spec line) did not change Claude's gloss picks (168 of 180 for both factual versions) or GPT-6.1-Sol's (59 vs 60 of 60), but gloss picks fell to 45 of 60 on Gemini 3.1 Pro and 32 of 60 on Gemini 3.8 Flash.", "source_title": "Cart for Agents: vague vs specific product facts simulation (method)", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "For gloss, the two factual versions differed in format (prose vs spec line) as well as the limit line, so the drop cannot be credited to the limit alone. No brief tested whether the limit helped an agent rule the product out correctly." }, { "text": "Every reports a controlled experiment in which fresh agent sessions picked from a fixed, shuffled set of supplied sources, and an article's citations changed sharply when its headline said what the article contained. Our design copied that fresh-session, shuffled, pick-two method.", "source_title": "How to Get AI Agents to Recommend Your Articles (Every)", "source_url": "https://every.to/also-true-for-humans/how-to-get-ai-agents-to-recommend-your-articles", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Author-reported simulation about articles, not products; the date is our check date. It is not a live search ranking study." }, { "text": "Researchers behind the ACES sandbox report that which product an AI agent selects varies sharply by model and model version, and that randomized listing position changes selection.", "source_title": "What Is Your AI Agent Buying? (arXiv v3)", "source_url": "https://arxiv.org/html/2508.02630v3", "source_date": "2025-12-15", "method": "controlled-simulation", "limitation": "Controlled sandbox with small assortments; selection shares are not market shares, and it does not measure deployed assistants." }, { "text": "OpenAI says ChatGPT picks shopping results by relevance to the shopper's query and context.", "source_title": "Shopping with ChatGPT Search (OpenAI Help Center)", "source_url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "source_date": "2026-10-27", "method": "documented", "limitation": "Living help page; the date is our check date. It states inputs, not ranking weights, and our API calls do not measure the ChatGPT product." } ], "sources": [ { "title": "Cart for Agents: vague vs specific product facts simulation (method)", "url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method", "accessed": "2026-10-27" }, { "title": "How to Get AI Agents to Recommend Your Articles (Every)", "url": "https://every.to/also-true-for-humans/how-to-get-ai-agents-to-recommend-your-articles", "accessed": "2026-10-27" }, { "title": "What Is Your AI Agent Buying? (arXiv v3, 2025-12-15)", "url": "https://arxiv.org/html/2508.02630v3", "accessed": "2026-10-27" }, { "title": "E-GEO: A Testbed for Generative Engine Optimization in E-Commerce (arXiv v2, 2026-07-14)", "url": "https://arxiv.org/html/2511.20867v2", "accessed": "2026-10-27" }, { "title": "Shopping with ChatGPT Search (OpenAI Help Center)", "url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search", "accessed": "2026-10-27" } ], "limitations": [ "Supplied-snippet simulation: models picked two of eight listings we gave them. Nothing was searched, ranked, opened, bought or published, so this says nothing about discovery, live visibility or sales.", "The test products, their brand and their facts are invented. Do not copy the numbers; never publish a fact you have not verified.", "One shelf of seven real competitors per category, three shopper briefs and six models called through APIs at low reasoning effort. API answers do not measure ChatGPT, Gemini apps or Google AI Mode.", "The factual versions disclosed information the vague versions hid. This does not show that rewording the same information changes picks.", "No brief required something the product could not do, and there was no \"none of these\" option, so the test cannot show whether stated limits help agents reject a wrong product.", "Gemini 3.1 Pro and Gemini 3.8 Flash had failed or cut-off replies; these are excluded, leaving 346 of 360 and 356 of 360 valid held-out sessions." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/guides/agent-commerce-optimization", "https://cartforagents.com/research/quiet-humidifiers", "https://cartforagents.com/guides/agent-ready-product-data", "https://cartforagents.com/guides/humidifier-field-guide" ], "published": "2026-10-27", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" }, { "id": "F02", "url": "https://cartforagents.com/research/quiet-humidifiers", "title": "Quiet vs 21 dB: which humidifier facts reach AI shopping agents", "description": "Every noise statement on a humidifier shelf in Shopify's agent catalog, traced to the field it came from: which listings gave a decibel figure and which only said quiet.", "answer": "In the top 10 results Shopify's agent catalog returned for \"humidifier for dry winter air large room quiet top fill\" on October 26, 2026, 4 product descriptions stated a decibel figure, 2 more listings had one only in Shopify-generated metadata, and no title had one; none of the figures came with a setting, distance or test method. The \"21dB Quiet\" listing gives a comparable number where its neighbour gives an adjective.", "take": "\"Quiet\" is an adjective; \"21 dB\" is evidence. If your product is quiet, print the number with its setting, because a shopper's agent can compare a number and can't compare a feeling.", "summary": "One humidifier listing says \"21dB Quiet\". Another says it \"runs quietly enough to leave on overnight\". Here is every noise statement on that shelf, traced to the field it came from.", "claims": [ { "text": "Both listings appeared in the same top 10 for \"humidifier for dry winter air large room quiet top fill\". In its description, one stated \"21dB Quiet\"; the other said it \"runs quietly enough to leave on overnight\".", "source_title": "Cart for Agents: quiet humidifiers, how we checked", "source_url": "https://cartforagents.com/research/quiet-humidifiers#method", "source_date": "2026-10-26", "method": "observed", "limitation": "Quoted from one catalog response, not measured by us. Store names and the raw response are withheld." }, { "text": "A decibel figure appeared in the description of 4 of the 10 listings (positions 2, 3, 5 and 7: 21, 26, 28 and 26 dB). Counting Shopify-generated metadata as well, 6 of 10 listings carried one (positions 6 and 9, both 26 dB, only in generated metadata). No title carried one.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "One search, one unpersonalized snapshot. The 4 counts the description field only; the 6 adds Shopify-generated metadata. It counts what was written, not how loud any unit is." }, { "text": "None of the four decibel figures in descriptions named a setting, a measuring distance or a test method. The only qualifier on any decibel figure in the ten records was \"as low as 26 dB\", which appeared in Shopify-generated metadata for position 6, whose description says nothing about noise.", "source_title": "Cart for Agents: quiet humidifiers, how we checked", "source_url": "https://cartforagents.com/research/quiet-humidifiers#method", "source_date": "2026-10-26", "method": "observed", "limitation": "Covers the fields in this catalog response. Product pages, manuals or spec sheets may state conditions we did not receive." }, { "text": "Position 2's description gave \"21dB\", but Shopify's generated spec list for that record had no noise line. The same list carried two entries reading \"Not specified\" (typical power consumption and mounting type).", "source_title": "Cart for Agents: quiet humidifiers, how we checked", "source_url": "https://cartforagents.com/research/quiet-humidifiers#method", "source_date": "2026-10-26", "method": "observed", "limitation": "One record. We cannot see how Shopify builds the list, so we do not know why the figure was left out." }, { "text": "Position 9's description said only \"low noise level\", while its generated metadata said \"Noise Level: 26 dB\" and \"Ultra-quiet operation at 26 dB\". Its generated spec list also gave runtime as \"Up to 25 hours (low mode)\", where the description said \"up to 25 hours\" with no setting.", "source_title": "Cart for Agents: quiet humidifiers, how we checked", "source_url": "https://cartforagents.com/research/quiet-humidifiers#method", "source_date": "2026-10-26", "method": "observed", "limitation": "The generated values may come from merchant data we did not receive, such as product fields or manuals. We cannot say whether they are correct." }, { "text": "Position 4 described noise only as \"runs quietly enough to leave on overnight\" and carried no generated metadata, but its description tied its runtime to a setting (\"up to 36 hours on low\"). Of the 8 descriptions that gave a runtime, it was one of 2 that named a setting; the other, position 8, said its run time was \"based off of running the unit on low setting\".", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Description field only. Neither statement is a measurement by us." }, { "text": "Position 5's description said \"up to 32 hours\" while its generated spec list said \"Up to 36 hours\".", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "observed", "limitation": "Two runtimes for one product in one record. We do not know which is right." }, { "text": "One description stated a 6.2 L tank, 320 ml/h mist output and up to 62 hours of runtime. At 320 ml/h a 6.2 L tank lasts about 19 hours, so 62 hours implies a lower average output that the description does not name.", "source_title": "Cart for Agents: receipts cited by our guides", "source_url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "source_date": "2026-10-26", "method": "inferred", "limitation": "Arithmetic on stated figures only (6.2 L divided by 0.32 L/h). Real output varies with setting, humidity and water level; we did not test the unit." }, { "text": "Shopify marks the description, options, metadata.attributes, metadata.tech_specs, metadata.top_features, metadata.unique_selling_points and variant condition fields as Inferred: generated or enriched by Shopify, possibly absent or varying in accuracy, and to be treated as discovery signals rather than merchant-authored source text.", "source_title": "Shopify: Global Catalog MCP, Inferred response fields", "source_url": "https://shopify.dev/docs/agents/catalog/global-catalog", "source_date": "2026-10-27", "method": "documented", "limitation": "Living developer documentation; the date is our check date. It does not say how the fields are generated or how any assistant uses them." }, { "text": "In a supplied-snippet simulation, an invented humidifier with vague copy was picked in 3 of 180 Claude sessions (three models), 0 of 60 on GPT-6.1-Sol, 6 of 60 on Gemini 3.8 Flash and 3 of 54 on Gemini 3.1 Pro, where random picking would include it 25% of the time. With its facts as a spec line it was picked in 134 of 180, 20 of 60, 19 of 59 and 15 of 55.", "source_title": "Cart for Agents: Vague vs specific product facts, simulated picks by Claude, GPT and Gemini models", "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation", "source_date": "2026-10-27", "method": "controlled-simulation", "limitation": "Models chose from snippets we supplied; the target product was invented. On GPT and Gemini the factual version won only with a detailed shopper brief that was written to fit it. Not live ChatGPT or Gemini results, visibility or sales." } ], "sources": [ { "title": "Cart for Agents: quiet humidifiers, how we checked", "url": "https://cartforagents.com/research/quiet-humidifiers#method", "accessed": "2026-10-27" }, { "title": "Cart for Agents: receipts cited by our guides", "url": "https://cartforagents.com/research/agent-shopping-research#receipts-cited-by-our-guides", "accessed": "2026-10-27" }, { "title": "Shopify: Global Catalog MCP", "url": "https://shopify.dev/docs/agents/catalog/global-catalog", "accessed": "2026-10-27" }, { "title": "Cart for Agents: Vague vs specific product facts, simulated picks by Claude, GPT and Gemini models", "url": "https://cartforagents.com/research/vague-specific-product-facts-simulation", "accessed": "2026-10-27" } ], "limitations": [ "One search, one unpersonalized Shopify agent-catalog response, top 10, taken October 26, 2026. It is not a sample of humidifiers, and six of the ten listings came from one seller.", "A Shopify catalog response is not a ChatGPT, Copilot or Gemini answer. We cannot see what any assistant showed a shopper, and nothing here measures rejection, ranking or recommendation lift.", "Every figure is a seller or Shopify statement. We measured no humidifier, so a decibel figure is not verified loudness, and a missing figure is unknown, not evidence that a product is loud.", "Shopify marks both the description and the generated metadata as Inferred, so neither is guaranteed to be the merchant's own words.", "Store names and the raw catalog response are withheld. Quotes are kept short." ], "related": [ "https://cartforagents.com/research/agent-shopping-research", "https://cartforagents.com/guides/agent-ready-product-data", "https://cartforagents.com/research/vague-specific-product-facts-simulation", "https://cartforagents.com/guides/humidifier-field-guide", "https://cartforagents.com/research/ai-product-data-unit-normalization", "https://cartforagents.com/research/missing-product-review-data" ], "published": "2026-10-26", "updated": "2026-10-27", "checked": "2026-10-27", "kind": "research" } ] }