There’s no screen Shopify actually calls the “Agentic tab”. What merchants mean is Sales channels > Agentic in the Shopify admin. Its search preview shows the raw output of Shopify Catalog search for whatever query you type. Next to each product sits a Listing quality meter built from five signals: description word count, number of images, verified reviews, variant and option completeness, and store policies.
Read it as a view of Shopify’s search, not of ChatGPT. Shopify itself says assistants “often re-rank results based on their own logic” and calls the preview “a directional signal, not an exact prediction”.
This walkthrough comes from Shopify’s public help pages, checked October 27, 2026. We haven’t seen your admin and don’t assume access to it.
What’s in the Agentic section
Shopify lists seven jobs the section does. Here’s each one with what it can and can’t tell you.
| Panel | What Shopify says it shows | What it can tell you | What it can’t |
|---|---|---|---|
| Sources > Shopify Catalog | Product count, or an Ineligible reason | Whether the store is in Shopify Catalog at all | Why one product is missing (eligibility checklist) |
| Channel controls | Which AI channels can use your Catalog data; direct checkout settings | Which routes are switched on | Whether any channel shows your products |
| Product data mapping | Where Catalog takes your title and description from; variant grouping | What Catalog receives | How an assistant displays it. Shopify says the mapping preview “isn’t an exact indicator” |
| Search preview | Raw Shopify Catalog search output for your query | Whether your products compete in Catalog search | The assistant’s final order, wording or pick |
| Listing quality and Listing insights | A coloured meter and five signals per product | What’s thin on your own listing | Relevance, popularity or anything the assistant adds |
| Performance | Sales, orders, sessions and conversion per AI channel | What actually came in | Which answers you appeared in |
| Knowledge base | Store information for agents | What you’ve told agents about policies | Whether an agent used it |
The bold rows are the ones people over-read. The rest of this guide is about them.
Walkthrough: the search preview, step by step
These steps follow Shopify’s help page. The labels in quotes are Shopify’s wording. We haven’t verified the screen, so expect small differences.
1. Run a query. Shopify describes three ways in: type a product name or keyword into “Enter a search query to see how you might rank”, or click a query under “Top search queries” or “Queries where your products appeared”.
How to read it: type the way shoppers talk to assistants, not the way they type into Google. “Humidifier” tests almost nothing. “Humidifier for a large bedroom, quiet, top fill” tests your facts.
2. Read the headline message. If any of your products are in the global top results, Shopify shows a message such as “10 of your products are in the top ranking search”. Shopify describes those as competitive against all other Shopify Catalog products for that query.
How to read it: this is a count of your products in Catalog’s top results for one query, at one moment. It isn’t a share of anything an assistant showed.
3. Open “Your products that could rank for this query”. Each product appears with its Listing quality indicator and Listing insights.
How to read it: Shopify’s phrasing is “could rank”. Keep that word.
4. Review the products that didn’t rank. If none of your products make the top 10, Shopify shows the products in your store that were most relevant to the query, with their insights. It points to fixes such as missing descriptions, too few relevant images or incomplete variant data.
How to read it: Shopify notes that a query with no real relevance to your store still returns your “most relevant” products, so odd results don’t mean the tool is broken.
5. Optional: “Preview this search”. Shopify says clicking a channel name shows how the query “might display” in that platform’s AI chat interface.
How to read it: treat it as a mock-up. Shopify doesn’t say how it’s produced, and it isn’t a capture of a real conversation.
The Listing quality meter, signal by signal
Shopify says the meter is “a visual sign of how complete and competitive your product listing is for discoverability”. It adds that listing quality covers only what you directly control. Relevance also depends on popularity, customer engagement and brand recognition.
| Signal | What Shopify says it measures | What it doesn’t measure | What to check yourself |
|---|---|---|---|
| Description completeness | Word count of the description | Whether the words state a value a shopper can check | Does the description state the facts that answer your top customer questions, with units and conditions? |
| Image coverage | Number of product images | Whether the images show the thing a shopper asks about | Is there an image of the feature that’s hard to describe, such as the fill opening or the label? |
| Product reviews | Average rating and review count, from reviews “verified by trusted sources” | Which sources count. Shopify doesn’t name them | Does your rating appear where you expect? Missing ratings aren’t zeros |
| Variant and option completeness | Number of variants and options, their in-stock availability, and option names with hard-to-understand acronyms or numbers | Whether each variant’s facts are right | Are option names words a shopper would use? Is each size or colour in stock under its own variant? |
| Shop policy completeness | Whether shipping, returns, refund and other policies exist | What the policies say | Are they current, and do they answer the questions customers actually ask? |
The first row is where a full meter can mislead. A word count rewards length. A shopper’s agent needs a checkable fact.
Worked case: what a word count misses
Shopify’s meter isn’t public, and we can’t see your admin. But Shopify says the preview shows the raw output of Shopify Catalog search, and we have one saved Catalog search: “humidifier for dry winter air large room quiet top fill”, taken October 26, 2026 through Shopify’s public Global Catalog with no buyer context. We can’t say whether the admin preview runs the same search in the same context, because Shopify doesn’t say.
Here are rough stand-ins for three of the five signals, counted from the returned records. They are not Shopify’s scores.
| Position | Description, about | Decibel figure in the description | Noise wording in the description |
|---|---|---|---|
| 2 | 28 words | Yes | States a dB figure |
| 3 | 190 words | Yes | States a dB figure |
| 4 | 160 words | No | “runs quietly enough to leave on overnight” |
| 5 | 230 words | Yes | States a dB figure |
| 7 | 200 words | Yes | States a dB figure |
| 8 | 160 words | No | “ultra-quiet” |
Positions 1, 6, 9 and 10 ran between about 30 and 45 words with no decibel figure in the description. Two of them, positions 6 and 9, had one in Shopify’s generated spec list. So a decibel figure is in the description of 4 of the 10 listings, and in the description or Shopify-generated metadata of 6 of 10. None is in a title.
Two things stand out. Position 2 states its noise level in about 28 words. Positions 4 and 8 use roughly six times as many words and never give a number. By word count alone, 4 and 8 look more “complete”. For a shopper who asked for quiet, position 2 is the one whose record answers the question.
The other stand-ins didn’t separate these listings at all. Every one of the ten records came back with one image and all four policy links (privacy, refund, shipping and terms of service). A product-level rating appeared on only 3 of the 10. We don’t know whether a search record carries every image a merchant uploaded, and Shopify doesn’t say.
Our simulation points the same way. With supplied snippets, an invented humidifier with vague copy was almost never picked: 3 of 180 Claude sessions, 0 of 60 on GPT-6.1-Sol, 6 of 60 on Gemini 3.8 Flash and 3 of 54 on Gemini 3.1 Pro. With its facts as a spec line, it was picked in 134 of 180, 20 of 60, 19 of 59 and 15 of 55. Each model picked 2 of 8, so random is 25%. That’s a simulation with snippets we supplied, not live ChatGPT. It still suggests why a word count is a weak proxy: vague copy gets excluded, facts get you considered, and you win when your facts answer a stated need competitors’ listings don’t. The full write-up has the method.
Interpretation checklist
Run this every time you use the preview. It keeps a useful tool from turning into a scoreboard.
- Write down the query, the date and the time. Catalog results change, and a screenshot without a query is worthless.
- Test shopper-shaped queries, three to five words longer than a keyword, built from real customer questions.
- Record “top ranking” counts as Catalog evidence only. Write “Shopify Catalog preview”, never “ChatGPT ranking”, in any report.
- For each product that could rank, note the weakest of the five signals, then check the fact behind it, not the meter.
- Treat a full description-completeness bar as unverified until you’ve checked that the description states your key values with units and conditions.
- Check option names for codes and numbers a shopper wouldn’t say (Shopify says the meter flags these).
- Check reviews at the source. If the meter shows none but your page has many, the question is which source Shopify trusts. Ask Shopify; don’t guess.
- Treat “Preview this search” as a mock-up, not evidence of what a platform showed.
- Compare against performance data, not against the preview. Sales, orders and sessions per channel are what happened. Shopify says sales combine referral and direct-checkout sales, and that these analytics aren’t yet available for headless stores.
- Change one thing, then re-run the same query on a later date. A single before-and-after on one query is a hint, not a test.
What the preview can’t settle
Shopify says agentic storefronts re-rank, and its Catalog overview says each channel “controls the final ranking, wording, and shopping experience”. So the preview answers one question well: how does my product compete in Shopify Catalog’s own search for this query? It doesn’t answer whether ChatGPT recommended you, or why a shopper didn’t buy. Our research note on Catalog versus ChatGPT lists the limits to state before making that leap. The ChatGPT route guide covers what happens after Catalog.
How we checked
On October 27, 2026 we read Shopify’s help pages for managing agentic storefronts, customizing product data, and Agentic-plan setup. On the same day we reopened our saved Shopify Global Catalog response for “humidifier for dry winter air large room quiet top fill” (taken October 26, 2026, top 10, no buyer, country or currency context). For each record, a short script counted the words in the description field after removing HTML, the images in the media list, the policy link types on the seller, and whether a product-level or variant-level rating was present. Word counts are rounded and depend on how HTML is stripped. Shopify marks the description field Inferred, so it may not match the description in the merchant’s admin exactly.
Your next step
Open Sales channels > Agentic and run the one question your customers ask most, written the way they’d say it. For your best product in the results, write down its weakest Listing insight. Then open its description and mark whether it states the fact that answers that question, with a unit. If the meter is full and the fact is missing, fix the fact first.