What Is llms.txt? Should You Add It to Your Site?
llms.txt has been hyped as the new robots.txt for AI. Here's what it actually is, what it doesn't do, and whether it's worth your dev time in 2026.
The Hype Around llms.txt
In late 2024, a file called llms.txt started appearing across developer blogs with bold claims: "the new SEO standard for AI", "control what ChatGPT knows about your site", "rank higher in LLMs."
The reality is more nuanced — and more interesting than either the hype or the dismissals.
Here's what llms.txt actually is, what the evidence shows, and how to think about it practically.
What llms.txt Is
llms.txt is a plain text or Markdown file placed in your website's root directory (e.g., yourdomain.com/llms.txt). It's designed to give large language models — not search crawlers — guidance on how to interpret and process your site's content.
It was proposed by Jeremy Howard, co-founder of fast.ai, in September 2024.
Think of it as a cross between robots.txt and a sitemap, but written for AI models rather than search engines.
A basic llms.txt looks like this:
# Maximal Studio
> AI agency in Bangalore building custom AI tools, automation systems, and enterprise AI marketing.
## About
- [About Us](https://www.maximalstudio.in/approach): Our philosophy and service approach
- [Case Studies](https://www.maximalstudio.in/case-studies): Real-world AI deployments
## Tools
- [AI ROI Calculator](https://www.maximalstudio.in/tools/ai-roi-calculator): Free tool to estimate automation savings
- [LLM Cost Calculator](https://www.maximalstudio.in/tools/llm-cost-calculator): Compare API costs across models
## Blog
- [Blog](https://www.maximalstudio.in/blog): Guides on AI automation and LLM workflows
The format is intentionally minimal — a brief description of your site, followed by links to important sections with short descriptions.
What llms.txt Actually Does (and Doesn't Do)
This is where you need to be careful about separating signal from noise.
What it does:
- Provides structured context for AI models that choose to read it during indexing or retrieval
- Signals intent about how your content should be understood
- May help with models that explicitly support the spec (some open-source LLMs and research tools)
What it doesn't do (as of March 2026):
- Not used by ChatGPT — OpenAI has not implemented support for llms.txt
- Not used by Google Gemini — Google has explicitly stated it won't use llms.txt
- Not used by Perplexity — no confirmed support
- Not used by Claude — Anthropic has not announced support
- Not an official standard — it's a proposal, not a W3C spec or industry standard
Google's official position: "llms.txt won't be used in our systems."
This doesn't mean it's useless — but it does mean it is not a shortcut to AI visibility on any major platform right now.
Why It Still Might Be Worth Adding
Despite the lack of current support from major AI platforms, there are practical reasons to implement llms.txt:
1. Future-Proofing
The spec exists. Some AI models already read it. As the ecosystem evolves, having it in place means you benefit from adoption without needing to retrofit.
2. Internal AI Tooling
If your team uses LLMs internally — crawling your own site for support bots, documentation agents, internal search — llms.txt helps those systems navigate your content more accurately.
3. The Implementation Cost Is Low
A basic llms.txt takes 20 minutes to write. It doesn't break anything. If it provides any marginal benefit now or in the future, the ROI is positive.
4. Research Tools and API Access
DataForSEO, some brand monitoring tools, and custom AI pipelines do respect llms.txt. If you're building anything that crawls your own site via AI, this matters.
How to Create Your llms.txt
Basic Structure
# [Site Name]
> [One sentence description of what your site is]
[Optional extended description]
## [Section Name]
- [Page Title](URL): Brief description
- [Page Title](URL): Brief description
## [Another Section]
- [Page Title](URL): Brief description
Best Practices
Keep descriptions factual and specific:
# Maximal Studio
> AI development agency in Bangalore. We build custom AI tools, multi-agent systems, and automation workflows for enterprises.
## Services
- [Approach](https://www.maximalstudio.in/approach): Our methodology for productizing AI services
- [Case Studies](https://www.maximalstudio.in/case-studies): Enterprise AI deployments with documented outcomes
## Free Tools
- [AI ROI Calculator](https://www.maximalstudio.in/tools/ai-roi-calculator): Estimate yearly savings from workflow automation
- [LLM Cost Calculator](https://www.maximalstudio.in/tools/llm-cost-calculator): Compare per-token costs across GPT-4o, Claude, Gemini, Llama
- [AI Readiness Assessment](https://www.maximalstudio.in/tools/ai-readiness-assessment): Score your organisation's readiness to deploy AI
Also consider llms-full.txt:
The spec includes an optional companion file — llms-full.txt — that contains the full content of important pages in a single file, making it easier for AI to consume your site in one pass.
What Actually Moves the Needle in 2026
While llms.txt is worth a quick implementation, it's the lowest-leverage thing you can do for AI visibility. The moves that actually matter:
| Action | Impact | Effort |
|---|---|---|
| FAQ schema on all informational pages | High | Medium |
| Answer-first content structure | High | Medium |
| Third-party mentions on review platforms | High | High |
| Consistent entity definitions across web | High | Medium |
| llms.txt file | Low-Medium (future) | Low |
| Author bylines with credentials | Medium | Low |
| robots.txt allowing GPTBot | Critical (if blocked) | Low |
The robots.txt point deserves emphasis: if you're blocking GPTBot (ChatGPT's crawler), no other optimisation matters for real-time retrieval.
Check your robots.txt and add:
User-agent: GPTBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: ClaudeBot
Allow: /
The Honest Verdict
Add llms.txt. It takes 20 minutes, costs nothing, and signals forward-thinking technical hygiene.
Don't rely on it. It is not currently driving visibility on ChatGPT, Perplexity, or Google Gemini.
Focus on what works now: structured content, FAQ schema, third-party authority, and making your existing pages answer questions clearly and directly.
The spec will likely become more widely adopted as AI search infrastructure matures. Being early is fine. Treating it as a primary AI SEO strategy in 2026 is wishful thinking.