{
  "id": "R1",
  "url": "https://cartforagents.com/research/vague-specific-product-facts-simulation",
  "title": "Vague vs specific product facts: simulated picks by Claude, GPT and Gemini models",
  "description": "Vague listings were almost never picked; factual ones won when they answered the shopper's brief. Pick counts across Claude, GPT and Gemini models, by brief and product.",
  "answer": "In a supplied-snippet simulation with an invented humidifier and lip gloss, six models almost never picked the vague listings (0 to 10% of held-out sessions, against 25% for random picking), and picked the gloss with facts in its description in 82 to 100%. The factual humidifier was picked in 74% of Claude sessions and about 30% on GPT-6.1-Sol and two Gemini models: every time the shopper asked for the facts it stated, and never when the ask was simply a quiet humidifier for a bedroom.",
  "take": "In our tests, vague copy lost on every model. Facts won when they answered what the shopper asked and the neighbouring listings didn't, and that condition is the most useful thing a merchant can take from this experiment.",
  "summary": "Vague copy was shut out by every model we tried. Facts got the product back into the running. Whether they won depended on the model, the shopper's brief and what the listings beside it already said.",
  "claims": [
    {
      "text": "With vague copy, the invented humidifier was picked in 3 of 180 held-out Claude sessions, 0 of 60 on GPT-6.1-Sol, 6 of 60 on Gemini 3.8 Flash and 3 of 54 on Gemini 3.1 Pro. The vague lip gloss was picked in 0 sessions on every model (0 of 180 Claude, 0 of 60 on each other model). Random picking would include it in 25% of sessions.",
      "source_title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "source_date": "2026-10-27",
      "method": "controlled-simulation",
      "limitation": "Supplied-snippet simulation with an invented brand and invented product facts; models never searched, opened pages or bought anything. Claude counts pool three models (Haiku, Sonnet and Opus 5.5). Failed calls are excluded from n, which is why some cells are below 60."
    },
    {
      "text": "Lip gloss facts written into the description were picked in 168 of 180 held-out Claude sessions (93.3%, Wilson 95% CI 88.7 to 96.1%), 60 of 60 on GPT-6.1-Sol, 53 of 60 on Gemini 3.1 Pro and 49 of 60 on Gemini 3.8 Flash.",
      "source_title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "source_date": "2026-10-27",
      "method": "controlled-simulation",
      "limitation": "The gloss facts (clear, non-sticky, no shimmer, fragrance-free, 5 mL) are invented, and the factual version disclosed information the vague version hid. This measures disclosure of fit facts, not better wording of the same information."
    },
    {
      "text": "The humidifier facts written as a spec line were picked in 134 of 180 held-out Claude sessions (74.4%, CI 67.6 to 80.3%), 20 of 60 on GPT-6.1-Sol (33.3%, CI 22.7 to 45.9%), 19 of 59 on Gemini 3.8 Flash (32.2%) and 15 of 55 on Gemini 3.1 Pro (27.3%).",
      "source_title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "source_date": "2026-10-27",
      "method": "controlled-simulation",
      "limitation": "The non-Claude averages hide an all-or-nothing split by shopper brief (next claim). Model averages describe these sampled models, not whole model families or the consumer apps built on them."
    },
    {
      "text": "On GPT-6.1-Sol, Gemini 3.8 Flash and Gemini 3.1 Pro, the humidifier spec line was picked in every session with the detailed shopper brief (20 of 20, 19 of 19, 15 of 15) and in none of the sessions with a one-line ask or no shopper at all (0 of 40 on each model). Claude picked it in 59 of 60, 32 of 60 and 43 of 60 sessions for the same three briefs.",
      "source_title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "source_date": "2026-10-27",
      "method": "controlled-simulation",
      "limitation": "Small cells (about 20 sessions per model and brief). The detailed brief was written so that its needs matched the invented product, and no brief deliberately required something the product could not do."
    },
    {
      "text": "In the supplied humidifier shelf, four of the seven real competing listings stated a noise level in decibels: one at 21 dB, two at 26 dB and one at 28 dB, the same figure as our invented product. Only one competitor, priced above USD 90, stated a room size.",
      "source_title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "source_date": "2026-10-27",
      "method": "observed",
      "limitation": "Counts our own supplied snippets: titles, descriptions cut to 215 characters and three feature bullets. Full product pages may state more. The figures are seller statements, not measurements."
    },
    {
      "text": "In the spec-line sessions with a one-line ask or no shopper, GPT-6.1-Sol and the two Gemini models included the competitor stating 21 dB in 113 of 120 sessions, while the invented 28 dB product was never picked. Claude included the 28 dB competitor priced at USD 39.99 in all 120 of those sessions and also picked the invented product in 75 of them.",
      "source_title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "source_date": "2026-10-27",
      "method": "controlled-simulation",
      "limitation": "Counts of which listings were picked in the spec-line sessions. They show what was chosen, not why; we did not test changing the competitors or the invented product's noise figure."
    },
    {
      "text": "We infer that stating facts made the invented humidifier eligible, but it won only where its facts answered something no competitor answered (room size under the shopper's budget). Where competitors stated the same or a quieter noise figure for less money, the facts did not carry it.",
      "source_title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "source_date": "2026-10-27",
      "method": "inferred",
      "limitation": "An interpretation of pick patterns, not a tested mechanism. Brand familiarity, review counts and price differences are alternative or additional explanations that this design did not separate."
    },
    {
      "text": "Adding a \"not for\" limit (with the facts in a spec line) did not change Claude's gloss picks (168 of 180 for both factual versions) or GPT-6.1-Sol's (59 vs 60 of 60), but gloss picks fell to 45 of 60 on Gemini 3.1 Pro and 32 of 60 on Gemini 3.8 Flash.",
      "source_title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "source_url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "source_date": "2026-10-27",
      "method": "controlled-simulation",
      "limitation": "For gloss, the two factual versions differed in format (prose vs spec line) as well as the limit line, so the drop cannot be credited to the limit alone. No brief tested whether the limit helped an agent rule the product out correctly."
    },
    {
      "text": "Every reports a controlled experiment in which fresh agent sessions picked from a fixed, shuffled set of supplied sources, and an article's citations changed sharply when its headline said what the article contained. Our design copied that fresh-session, shuffled, pick-two method.",
      "source_title": "How to Get AI Agents to Recommend Your Articles (Every)",
      "source_url": "https://every.to/also-true-for-humans/how-to-get-ai-agents-to-recommend-your-articles",
      "source_date": "2026-10-27",
      "method": "controlled-simulation",
      "limitation": "Author-reported simulation about articles, not products; the date is our check date. It is not a live search ranking study."
    },
    {
      "text": "Researchers behind the ACES sandbox report that which product an AI agent selects varies sharply by model and model version, and that randomized listing position changes selection.",
      "source_title": "What Is Your AI Agent Buying? (arXiv v3)",
      "source_url": "https://arxiv.org/html/2508.02630v3",
      "source_date": "2025-12-15",
      "method": "controlled-simulation",
      "limitation": "Controlled sandbox with small assortments; selection shares are not market shares, and it does not measure deployed assistants."
    },
    {
      "text": "OpenAI says ChatGPT picks shopping results by relevance to the shopper's query and context.",
      "source_title": "Shopping with ChatGPT Search (OpenAI Help Center)",
      "source_url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search",
      "source_date": "2026-10-27",
      "method": "documented",
      "limitation": "Living help page; the date is our check date. It states inputs, not ranking weights, and our API calls do not measure the ChatGPT product."
    }
  ],
  "sources": [
    {
      "title": "Cart for Agents: vague vs specific product facts simulation (method)",
      "url": "https://cartforagents.com/research/vague-specific-product-facts-simulation#method",
      "accessed": "2026-10-27"
    },
    {
      "title": "How to Get AI Agents to Recommend Your Articles (Every)",
      "url": "https://every.to/also-true-for-humans/how-to-get-ai-agents-to-recommend-your-articles",
      "accessed": "2026-10-27"
    },
    {
      "title": "What Is Your AI Agent Buying? (arXiv v3, 2025-12-15)",
      "url": "https://arxiv.org/html/2508.02630v3",
      "accessed": "2026-10-27"
    },
    {
      "title": "E-GEO: A Testbed for Generative Engine Optimization in E-Commerce (arXiv v2, 2026-07-14)",
      "url": "https://arxiv.org/html/2511.20867v2",
      "accessed": "2026-10-27"
    },
    {
      "title": "Shopping with ChatGPT Search (OpenAI Help Center)",
      "url": "https://help.openai.com/en/articles/11128490-shopping-with-chatgpt-search",
      "accessed": "2026-10-27"
    }
  ],
  "limitations": [
    "Supplied-snippet simulation: models picked two of eight listings we gave them. Nothing was searched, ranked, opened, bought or published, so this says nothing about discovery, live visibility or sales.",
    "The test products, their brand and their facts are invented. Do not copy the numbers; never publish a fact you have not verified.",
    "One shelf of seven real competitors per category, three shopper briefs and six models called through APIs at low reasoning effort. API answers do not measure ChatGPT, Gemini apps or Google AI Mode.",
    "The factual versions disclosed information the vague versions hid. This does not show that rewording the same information changes picks.",
    "No brief required something the product could not do, and there was no \"none of these\" option, so the test cannot show whether stated limits help agents reject a wrong product.",
    "Gemini 3.1 Pro and Gemini 3.8 Flash had failed or cut-off replies; these are excluded, leaving 346 of 360 and 356 of 360 valid held-out sessions."
  ],
  "related": [
    "https://cartforagents.com/research/agent-shopping-research",
    "https://cartforagents.com/guides/agent-commerce-optimization",
    "https://cartforagents.com/research/quiet-humidifiers",
    "https://cartforagents.com/guides/agent-ready-product-data",
    "https://cartforagents.com/guides/humidifier-field-guide"
  ],
  "published": "2026-10-27",
  "updated": "2026-10-27",
  "checked": "2026-10-27",
  "kind": "research"
}