/
/
The Truth About llms.txt (From the People Actually Optimizing Ecommerce Sites for AI Agents)
Almost every brand we talk to eventually asks some version of the same question. "We keep hearing we need an llms.txt file. Is that what gets us recommended by ChatGPT?"
It's a fair question, and I get why it won't go away. The file sounds important, it sits in the same spot as robots.txt, and half the advice online treats it like a cheat code. So let me answer it the way I would on a call, before we get into the weeds. Yes, go ahead and put one up. No, it's almost certainly not doing what you were told it does. And if your team is hand-editing it every week, you're spending that time in the wrong place.
That's really the whole post. But the reason behind it matters, because it changes where you point your effort.
How we all got confused
llms.txt is a proposed standard. It's a short file that sits at the root of your website and hands an AI a simple map of your site: a sentence or two about who you are, plus links to the pages you want it to trust.
The idea is fine. The name is what caused the trouble. Because it sits right next to robots.txt and looks a lot like it, everyone reached for the nearest comparison. People started calling it "robots.txt for AI" or "a sitemap for AI search." Those comparisons quietly promised something the file never actually offered, which is that putting it up gets you crawled, indexed, or ranked. It does none of that, and the AI companies have gotten pretty clear about saying so.
What it does not do
It doesn't control which bots can visit you. That's still robots.txt. llms.txt doesn't allow or block anything, and no crawler needs it to reach your pages.
And it doesn't move you up in AI search. This is the costly myth, because it sends teams chasing a file instead of the things that actually decide whether an AI recommends your product. When you read the guidance from each platform side by side, it all points the same way. Google has said plainly that it does not use llms.txt for Search. OpenAI's advice for showing up in ChatGPT is about letting its crawler visit you, not blocking it by accident, and having content worth reading. llms.txt isn't on the list. Anthropic points you to its crawlers and robots.txt. Bing leans on other tools for keeping content fresh.
Here's the part that convinced me. OpenAI, Anthropic, and Perplexity all publish their own llms.txt files, but they use them for their developer docs. If this were a real lever for ranking the open web, the companies building the AI would be the last ones to quietly keep it for documentation. What that really tells you is what the file is for. It helps AI tools find their way around documentation, which is a very different job from getting an online store to show up.
The last bad habit worth calling out is the urge to cram everything into it. I've seen brands try to shove twenty thousand product links into one file. Don't. It makes the file useless and just repeats data that already lives on your product pages, which is where the AI goes for the real version anyway.
What it's actually good for
Take the bad comparisons away and llms.txt has a simple, honest job. It's a map that helps an AI find its way around your site. The newer version of the standard makes this clearer. It's a short description of your site plus links to the pages you want the AI leaning on, with new ways to help the AI even find the file to begin with. For big catalogs, the newer approach of using smaller files for different sections of the site makes a lot more sense than one giant file.
The way to picture it is easy. An AI lands on your site, and llms.txt helps it understand the place. That's it. It sits next to the part of the web that decides rankings, not on top of it. It doesn't get you found. It just helps whatever found you make sense of what it's looking at.
The one platform I wouldn't brush off is Perplexity. It publishes its own llms.txt and actually tells AI tools to go fetch it to find its documentation. That's real proof of the file being used in a live setting, which is more than most of this space can say. Even so, Perplexity's own advice says that showing up in its search comes from letting its crawler read your pages. So it's useful for AI tools reading your content, but there's no proof it helps you rank. Both of those are true at the same time.
So keep it, but let a machine write it
When brands ask if it's worth bothering with, my answer is yes, with two rules.
Don't touch it by hand. A file you edit yourself is out of date within a week of your next catalog change, and an old map is worse than no map at all. Have it built automatically from your store or product feed so it stays honest on its own.
And use it to lay out the shape of your site, not to dump your whole catalog into it. The good version reads like a clean table of contents:
Point to your main category and product pages, and let those pages carry the real data. The file shows the way. The pages hold the truth. In practice it's worth a small slice of your effort, maybe a few percent, as cheap insurance while the standard grows up.
Where the rest of your effort should go
If llms.txt is a rounding error, here's what isn't. When we look at a brand's AI visibility, this is roughly the order we work in, because it's the order that actually changes whether a product gets found and recommended.
First, make sure the AI bots can even reach you. That means OpenAI, Perplexity, Anthropic, Google, and Bing's crawlers aren't quietly getting blocked by your site settings or your security tools. You'd be surprised how often a brand's biggest AI problem is a single firewall rule.
Then the basics that everything else sits on. Product pages that can actually be found and read. Complete product information the page shows on its own, without needing extra code to run, so things like the name, brand, price, whether it's in stock, the sizes and colors, the product codes, images, and specs are all right there. And clean structured data so the AI can read all of that directly. After that, your shopping feeds, like Google Merchant Center, Bing's version, Shopify's feeds, and the new AI shopping feeds starting to show up.
Next is the content that builds trust. Pages that actually say who a product is for, how it stacks up against others, what it's made of, and what it works with. Signals that tell the AI when your prices and stock change, so those updates spread quickly. And backing from the outside world, like reviews, articles, and forum mentions that AI tools use to decide you're trustworthy. Then how easy your site is for an AI to use, like clean page structure, links that don't break, and support for checking out. And then, at the very bottom of the list, llms.txt.
That order isn't a knock on the file. It's just that the things above it decide whether your product info can be found and trusted in the first place, and llms.txt sits one step away from all of that.
Where Banavo lands on this
We put up and maintain llms.txt automatically for the brands we work with, because it's cheap, it's low-risk, and it keeps them ready as the standard grows. But we're careful about how much weight we give it. In how we score a brand's readiness, llms.txt counts for a point or two out of a hundred. It's a nice-to-have, not a must-have, and we won't mark a brand down hard for missing it. The weight goes where the results come from: whether the bots can reach you, your product data, your structured data, your feeds, how fresh everything is, and how much the outside world backs you up.
So if you take one thing from this, don't let anyone sell you on "llms.txt improves your AI rankings." What's actually true, and what we tell every brand, is simpler. We'll keep the small emerging standards like llms.txt handled for you, and put the real effort into the things that decide whether an AI agent puts your product on the shelf.
That's the less exciting answer. It's also the one that works.
Want to know how ready your product pages actually are for AI shopping agents? Run them through Banavo's PDP readiness checker at banavo.ai/pdp-readiness. It looks at the bigger picture of what actually drives your visibility, not just one file.
