We tested two things about our own article titles. First: when an AI model answers a merchant’s question from a list of search results, does it cite an article whose title says plainly what’s inside, or one with a clever headline? Second: when you show a model all 84 of our planned article titles, what does it pick to read itself, and what does it pick for a busy Shopify founder?
Plain packaging won the citation test on every model. Titles and descriptions that stated the article’s contents were cited in 405 of 432 Claude sessions, 144 of 144 on GPT-6.1-Sol and 143 of 144 on Gemini 3.8 Flash. The clever versions got 28 of 108, 5 of 36 and 15 of 36. The reading test produced two different shelves. For their own reading, the models favoured titles about how AI agents behave and how we audit our research. For the founder, they picked practical product-data titles, and almost never the AI-behaviour ones.
This is a supplied-snippet simulation with role-framed prompts. It shows how five models chose among snippets we wrote. It doesn’t show what humans want to read, what ChatGPT or Google show, or that models have tastes.
The citation test in one table
Each model saw eight search results for a merchant’s question: seven real results from the actual search page for that question, plus one planned Cart for Agents article in one of five packages. It answered in three or four sentences and declared the two sources it cited. Results were reshuffled for every session. The cell counts sessions in which our article was one of the two cited.
| Package | Claude (3 models) | GPT-6.1-Sol | Gemini 3.8 Flash |
|---|---|---|---|
| Clever (all three questions) | 28/108 (25.9%) | 5/36 (13.9%) | 15/36 (41.7%) |
| Literal, four formats pooled | 405/432 (93.8%) | 144/144 (100%) | 143/144 (99.3%) |
The literal formats were: exact contents, a question, a checklist, and exact contents plus the label “Cart for Agents”. The clever versions were test alternatives we wrote to be weak, not our real titles.
Most of the clever citations came from one question, where the real competing results were a weak match for the topic, so even a vague headline looked relevant. On the other two questions, clever packaging was close to invisible.
What the packages said
Here are the five packages for the first question, “What is agentic commerce, and what should a Shopify merchant do about it?”, with their citation counts.
| Package and title | Claude (n=36) | GPT (n=12) | Gemini Flash (n=12) |
|---|---|---|---|
| Clever: “The Cart Before the Click” | 1 | 0 | 4 |
| Exact contents: “Agentic Commerce for Merchants: Discovery, Recommendation, Seller Choice and Checkout” | 30 | 12 | 11 |
| Question: “What Is Agentic Commerce, and What Can Shopify Merchants Control?” | 35 | 12 | 12 |
| Checklist: “Agentic Commerce Checklist: Five Stages a Merchant Should Check” | 35 | 12 | 12 |
| Exact contents + “| Cart for Agents” | 29 | 12 | 12 |
The clever package’s description read: “A new shopping story is unfolding. Here is how merchants can find their place in it.” The literal descriptions named the stages and what the reader would check at each one. Nothing about the article differed except those two lines. None of these articles existed yet: we tested packages for planned articles, so the models were judging a promise, not reading a page.
Results by question
“What does an AI shopping agent do, and which parts can merchants control?”
| Package | Claude (n=36) | GPT (n=12) | Gemini Flash (n=12) |
|---|---|---|---|
| Clever: “Your Next Customer Might Not Be Human” | 0 | 0 | 0 |
| Exact contents | 34 | 12 | 12 |
| Question | 30 | 12 | 12 |
| Checklist | 33 | 12 | 12 |
| Exact contents + brand | 35 | 12 | 12 |
“How should merchants publish negative product attributes so AI shopping agents can rule out unsuitable products?”
| Package | Claude (n=36) | GPT (n=12) | Gemini Flash (n=12) |
|---|---|---|---|
| Clever: “The Power of Saying No” | 27 | 5 | 11 |
| Exact contents | 36 | 12 | 12 |
| Question | 36 | 12 | 12 |
| Checklist | 36 | 12 | 12 |
| Exact contents + brand | 36 | 12 | 12 |
Two things to take from the detail:
- No literal format won everywhere. On the first question, the question and checklist formats tied on Claude; on the second, exact contents and exact-plus-brand did best on Claude. On GPT and Gemini nearly every literal format hit the ceiling. Treat question, exact-contents and checklist titles as equally good candidates.
- The brand label didn’t buy trust. Adding “Cart for Agents” moved Claude by one citation in each direction on the first two questions. Our domain was visible in every version anyway, and we tested no author credentials.
Does the result hold under a stricter count?
The main count used the two source IDs each model declared. Some answers also cited extra sources in their prose: 89 of 540 Claude answers (87 of them from Haiku) and 23 of 180 Gemini Flash answers; none on GPT. A rough sentence count also flagged 40 Claude answers and 8 GPT answers outside the requested three or four sentences.
So we recounted two ways: counting our article if it was cited anywhere in the prose, and counting only answers that followed the format exactly. On Claude, the first question’s clever package went from 1 declared citation to 2 in the prose, and 1 of 30 format-compliant answers; the literal packages were cited in at least 80% of the answers under every count. The clever-versus-literal gap held on every model under both recounts. Those subsets have uneven sizes, so they’re a check, not a new result.
The reading test: same 84 titles, two roles
In a separate test, each session showed all 84 of our planned article titles, shuffled, with no descriptions, and asked for five. Each session used one of two prompts:
- Own reading: “You are an AI agent with some free time to read. Below are articles from Cart for Agents about AI shopping and merchant product data. Pick the five you would most want to read.”
- For a founder: “Your user is the founder of a 15-person Shopify brand selling spec-heavy products. They want practical steps they can act on this month. Below are articles from Cart for Agents. Pick the five you would send them.”
Before running anything, we fixed a group of six titles about how agents behave: how agents choose products, constraint coverage, comparison cost, coverage measurement, binding facts to the right variant, and prompt injection. The question was how much of each shelf that group took.
| Model | Own reading | For the founder | Difference (95% interval) |
|---|---|---|---|
| Claude (3 models, 90 sessions per role) | 120/450 picks (26.7%) | 3/450 (0.7%) | 26.0 points (22.9 to 28.9) |
| GPT-6.1-Sol (30 sessions per role) | 72/150 (48.0%) | 0/150 (0%) | 48.0 points (44.0 to 52.0) |
| Gemini 3.8 Flash (30 sessions per role) | 92/150 (61.3%) | 3/150 (2.0%) | 59.3 points (54.7 to 64.0) |
The intervals come from resampling whole sessions, because five picks in one session aren’t independent.
We also fixed a broader group in advance that added eight research and measurement titles. On GPT it took 26 of 150 founder picks, all of them from one title, “Measure the fix: a controlled before-and-after product test”, which GPT sent the founder in 26 of 30 sessions. That’s a practical method, and it shows the line between “research” and “useful” isn’t clean.
The paired shelf, title by title
Each cell is the number of sessions in which that title was one of the five picks: own reading first, founder second. Claude has 90 sessions per role; GPT and Gemini Flash have 30. Titles are as they stood on the test date; some have since changed.
| Title | Claude (own / founder) | GPT (own / founder) | Gemini Flash (own / founder) |
|---|---|---|---|
| Product specifications with units: make numbers comparable | 0 / 82 | 0 / 23 | 0 / 30 |
| Spec-sold products: build an agent-ready category data model | 0 / 62 | 0 / 30 | 0 / 30 |
| Agent-ready product data: the factual record every SKU needs | 14 / 61 | 0 / 0 | 9 / 11 |
| Shopify metafields and taxonomy: what might reach the agent catalog | 1 / 17 | 0 / 25 | 0 / 30 |
| Shopify and ChatGPT: what is automatic and what you still control | 1 / 42 | 0 / 9 | 0 / 21 |
| Measure the fix: a controlled before-and-after product test | 6 / 7 | 0 / 26 | 0 / 0 |
| The product evidence contract: every claim needs value, scope and source | 60 / 16 | 28 / 0 | 16 / 0 |
| How AI shopping agents choose products: evidence, not ranking folklore | 50 / 0 | 7 / 0 | 30 / 0 |
| Prompt injection in product pages: what merchants should never publish | 30 / 0 | 28 / 0 | 19 / 0 |
| Agent comparison cost: expose facts without five tool calls | 3 / 0 | 20 / 0 | 30 / 0 |
| Minus 30 vs -30°C: the matcher error that changed our research | 59 / 0 | 4 / 0 | 0 / 1 |
| We retired our first agent gap score: here’s the audit trail | 57 / 0 | 0 / 0 | 0 / 0 |
Read across a row and the two roles barely overlap. Two founder picks held on every model: specifications with units, and a category data model. GPT and Gemini also leaned hard on which Shopify fields reach the agent catalog. The own-reading shelf was not consistent. Claude leaned toward our candid research stories, GPT toward security and evidence rules, and Gemini toward how agents choose and compare. The most-picked title for Claude’s own reading, the evidence contract, wasn’t in the behaviour group. We left it out rather than redraw the group after seeing results.
What this means for packaging a merchant article
These results shaped how we title this site. Our rules, and why:
- Say what’s inside, in the title and the description. Name the topic and the merchant job, then list the actual ingredients: stages, fields, checks, a worked example. In this test that did far better than a clever line on every model, and Every saw the same kind of change in its own experiment.
- Pick any literal format. Exact contents, a question and a checklist all performed at or near the ceiling. Choose the one that reads best for a person.
- Don’t lean on brand or a year. The brand label made no clear difference. We didn’t test a year here, but in our product test a “New 2026 / Bestseller” label didn’t rescue vague copy (0 of 108 Claude screening sessions per category). See Vague vs specific product facts.
- Lead with the founder shelf. The practical spine of this site is specifications with units, the per-SKU record and the category data model, because that’s what every model sent a founder. Our research and audit stories stay as trust pieces. Their appeal to models reading for themselves is not evidence that founders want them.
Only three article topics were tested. For the rest of our titles this is a working editorial rule, not a measured win.
How the test worked
We followed the design Every used to test article headlines: a fresh session for every answer, a fixed set of supplied results in a newly shuffled order, and a count of how often the target was chosen. (Every’s experiment.)
- Citation test. Three planned articles, five packages each. Seven real results from the original search page for each question, kept word for word and reshuffled with our package in every session. One thin user ask per article. Claude: 12 sessions per package per model across Haiku, Sonnet and Opus 5.5, 540 valid sessions. GPT-6.1-Sol and Gemini 3.8 Flash: 12 per package, 180 valid sessions each.
- Reading test. All 84 planned titles, unmodified, no descriptions, shuffled, five picks. 30 sessions per role per model: 180 Claude sessions, 60 on GPT and 60 on Gemini Flash, all valid.
- Settings. One message per session, no tools, no browsing, no system persona, low reasoning effort. Display IDs were assigned after shuffling, so no ID gave our article away. Code checked every returned ID; no model graded another.
- Fixed in advance. Hypotheses, sample sizes, the six-title behaviour group and the broader group were written down before any model was called. The GPT and Gemini runs reused the same prompts and snippets.
- Position check. Shuffling spreads position effects; it doesn’t remove them. The first-half versus last-half gap for our article in the citation test was 6.5 points on Claude, 4.4 on GPT and 4.6 on Gemini Flash. These are diagnostics, not corrections.
- Intervals. Wilson 95% intervals for counts; session-bootstrap intervals for the reading-shelf differences. They describe repeated prompts to these models, not a sample of people or of all AI agents.
- Spend. These tests ran in the same round as our product-facts simulation. The whole Claude round cost US$17.03 in known API charges; the GPT and Gemini replication cost US$5.68 known.
What this does not show
- It isn’t live search. The models never searched, ranked or opened anything. Real assistants retrieve pages first, may rewrite the title and snippet they show, and may read the article itself. What is and isn’t known about live selection is sorted in How shopping agents choose products.
- It isn’t human demand. No people read anything. A model asked to pick for “a founder” is imagining one.
- It isn’t proof that agents have tastes. “Free time to read” is a role we assigned in the prompt. Every reported that agents’ free-reading choices had no correlation (0.00) with its human popularity ranking; we have no human data at all.
- It doesn’t separate title from description. Both changed together in the citation test.
- It doesn’t rank formats. The cells are small (12 sessions per model), and many literal packages hit 100%.
- It doesn’t cover your topic. Three questions, one set of competing results each.
Your next step
Take one article or product page you rely on and read its title and meta description aloud. If someone who saw only those two lines couldn’t say what’s inside, rewrite them to name the topic, the job and the main ingredients. Keep a copy of the old version and the date, so you can compare later.