Three ways an AI model learns about your products

Ask an AI model about a product you launched a few months ago, and it may describe the one you sold last year. Ask again and let it search the web, and it can read your new page and get the details right.

The two answers came from different places. An AI model learns about a store’s products in three ways. It remembers what it read before its training ended. It looks pages up when a shopper asks. And it receives product data straight from the store, which for a Shopify store happens through Shopify. Knowing which route an answer came from tells you whether you can change it, and where.

The piece takes the three routes in turn. Then it turns to the file on your store that lets each AI company’s software in or keeps it out, and it ends with four checks you can run yourself. The examples follow Tarrowby, a made-up store that sells travel bags.

What does an AI model already know about your store?

It knows what it learned in training, and that stops at a date. An AI model is trained on a vast amount of text collected up to a certain day. After that it learns nothing new until its maker trains a newer version. Anthropic, which makes Claude, publishes that date for each version and calls it the reliable knowledge cutoff (opens in a new tab). Every version of Claude on offer in October 2026 learned from text up to June 2026.

Memory is good at what has been written about for years, such as which brands are known for what and what reviewers said about a bestseller. It is poor at anything recent. A coffee roaster that changed a blend last month, or a skincare brand that reformulated a cream, will find an AI model’s memory still describing the old one.

Memory counts even when the AI model searches. When ChatGPT picks products to show, OpenAI says (opens in a new tab) one of the things it weighs is the answer it would have given before looking at any new search results.

Pages reach that memory through a crawler, a piece of software an AI company sends round the web to read websites and copy what it finds. OpenAI (opens in a new tab) calls its training crawler GPTBot, and Anthropic (opens in a new tab) calls its own ClaudeBot. Perplexity (opens in a new tab) says it sends no crawler for training. Google uses a setting instead, which comes up further down. Whatever these crawlers collect today can shape only future versions.

What does it look up when a shopper asks?

It searches the web as it stands that day. According to OpenAI (opens in a new tab), ChatGPT can search on its own when a question would benefit from current information. A question about what to buy usually does, since prices and stock change all the time.

When it searches, it looks in an index, a vast collection of pages that a search crawler has already read and filed so they can be found quickly. Each AI company keeps its own. OAI-SearchBot builds ChatGPT’s, PerplexityBot builds Perplexity’s and Claude-SearchBot builds Claude’s. ChatGPT also sends searches to partner search companies, and OpenAI’s help page (opens in a new tab) points to Microsoft, which runs Bing, and to Shopify as two of them.

Google runs its AI features on the search it already has. AI Overviews and AI Mode draw on the same index as ordinary Google Search, which Googlebot builds. Google says (opens in a new tab) a page needs only to be in that index and able to show a snippet, the short preview under a search result. The Gemini app is handed pages from the same index when it answers.

An AI model can also open a single page at the moment someone asks, for example when a shopper pastes in a link to your product and asks about it. Those visits go out under the names ChatGPT-User, Claude-User and Perplexity-User.

This is the route that picks up a change. Once a crawler has read your new page, the next answer can use it, which is how the second Tarrowby answer gets the new set right. Which searches an AI model runs, and which of the pages it finds it trusts, is the subject of How AI search decides which brands to recommend.

What does your store send AI models directly?

It sends your product data, and on Shopify that happens without any work from you. Shopify has a feature for selling inside AI apps, called agentic storefronts (opens in a new tab), and it is switched on by default for eligible stores. Through it, Shopify passes your product details to AI channels including ChatGPT, Google AI Mode, the Gemini app and Microsoft Copilot. You can see which ones are on in your Shopify admin under Sales channels, then Agentic.

OpenAI confirms (opens in a new tab) that a Shopify store’s product data already reaches ChatGPT this way, and that the store owner needs to do nothing more. ChatGPT may write a simpler title and description of its own from what it receives, because different stores word the same product in different ways. Shopify says (opens in a new tab) prices and stock stay in step across these channels, and OpenAI notes that a price change can take a little while to show in ChatGPT.

Google has a route of its own. Shopify’s Google & YouTube channel (opens in a new tab) copies your products into Google Merchant Center, where sellers hand Google their product details. From there they join the Shopping Graph (opens in a new tab), Google’s record of products for sale. It held more than 50 billion listings in 2025, and AI Mode draws on it (opens in a new tab) for shopping questions.

The titles and descriptions you write in Shopify travel by this route as well as on your product pages, so a careful rewrite reaches both.

How do you tell each crawler yes or no?

Your store answers every crawler in one short text file, robots.txt, found at your store’s address followed by /robots.txt. Each section of the file names a crawler and says which pages it may read. A section that says User-agent: GPTBot followed by Disallow: / tells GPTBot it may read none of your pages.

Shopify gives every store a default file (opens in a new tab) that lets crawlers read its public pages and names none of the AI crawlers. Changing it means adding a template called robots.txt.liquid to your theme. Shopify warns that a wrong line can cost a store its search traffic, so the edit is a job for whoever looks after your theme.

Google is arranged differently. Google says (opens in a new tab) AI is built into Google Search, so the section for Googlebot covers ordinary results and AI Overviews together. This file cannot let one in and keep the other out. Google-Extended (opens in a new tab) is a name with no crawler of its own. It decides whether Google may train Gemini on your pages, and whether the Gemini app may use them when it answers. Google says it has no effect on Google Search.

The other names can each be answered separately. OpenAI describes (opens in a new tab) each of its settings as independent of the others, so a store can refuse its training crawler and still let its search crawler in.

What does saying no cost?

Refusing a search crawler takes your pages out of that AI model’s answers. OpenAI says (opens in a new tab) a site that refuses OAI-SearchBot will not be shown in ChatGPT’s search answers. Anthropic says (opens in a new tab) refusing Claude-SearchBot may make a site less visible in Claude’s. OpenAI and Perplexity (opens in a new tab) each say a change to the file takes about a day to reach their systems.

Refusing a training crawler keeps your pages out of what future versions learn from. Refusing Google-Extended goes further. Your pages stay out of Gemini’s training and also out of the pages the Gemini app is handed when it answers, while Google Search carries on as before.

The crawlers that open a page for a person follow looser rules. OpenAI says robots.txt may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores it, since a person asked for the page.

None of these lines affects the product data Shopify sends. Shopify says (opens in a new tab) blocking AI crawlers in robots.txt does not stop Shopify sending your products to the AI channels you have switched on. Those are managed in the admin.

Should you let AI companies train on your pages?

Of all the lines in the file, this is the one worth deciding on purpose. News publishers moved early, because their writing is what they sell. By the end of 2023, the Reuters Institute found (opens in a new tab), 48 per cent of the most widely used news sites in ten countries were refusing OpenAI’s crawlers.

A store’s pages have a different job. They describe products to people who might buy them, and an AI model that has learned from them can describe those products in the store’s own words. A store that refuses is still known through what other sites write about it, and those sites become the only pages about it that future versions learn from.

The decision is the store’s. What matters is that someone at the store makes it and checks that the file says so.

How can you check your own store?

Four checks show how each route sees your store today. You need a browser and your Shopify admin.

1. Type your store’s address followed by /robots.txt and look for the names in Fig. 2. A Disallow: / under one of them is a refusal, and a section headed User-agent: * applies to every crawler at once. Shopify’s default keeps crawlers out of private pages such as the cart and checkout, which is normal.
2. In your Shopify admin, open Sales channels, then Agentic, and see which AI channels are switched on. If you sell through Google, look at the Google & YouTube channel too.
3. Search Google for one product’s full name. A result with a short preview can be used in AI Overviews and AI Mode. Try Bing as well, since ChatGPT sends some of its searches to Microsoft.
4. Ask Claude about something you changed this year and tell it not to search the web (opens in a new tab). Then ask again and tell it to search. The difference between the two answers is the gap between its memory and your page.

Getting read is where it starts. Which of the pages an AI model trusts, and why one store is named in an answer while another is left out, is covered in How AI search decides which brands to recommend.

Products change, and every route that carries their facts has to keep up. Indaeo AI and the Indaeo team take that work on for your store. Indaeo AI puts your shoppers’ questions to five AI models every day and reads what each answer says about your products, so an out-of-date fact shows up. Its Product agent writes the text of each product page from your own sources, including the title and description that Shopify sends on. The Indaeo team sees each change through with you.