Try this first · freeCheck your robots.txt file
Every website can have a small file that tells bots where they can and can't go. Some sites block AI bots without the owner knowing, often from a security plugin or an old setting.
Do it todayType yourwebsite.com/robots.txt into your browser (with your real web address). If you see an AI bot's name, like GPTBot or PerplexityBot, followed by Disallow: /, that bot is told to stay out. Ask whoever runs your site to remove it.
Your bonusThe 10 things AI needs to read your site
- Your robots.txt isn't blocking AI. Open yourwebsite.com/robots.txt. Look for AI bot names like GPTBot, OAI-SearchBot, PerplexityBot or ClaudeBot next to "Disallow." That line tells the bot to stay out.
- Your words are in the page itself. Right-click your page and pick "View Page Source." Search for a sentence from your page. If it isn't there, it only shows up after code runs, and many AI bots don't run that code.
- No "noindex" by mistake. In that same page source, search for "noindex." If your important pages have it, you're telling search engines to skip them.
- You have a sitemap, and it's submitted. Check yourwebsite.com/sitemap.xml. Submit it in Google Search Console and Bing Webmaster Tools.
- Pages load fast and don't break. No error pages, no login walls in front of the info people need, and it works on a phone.
- One clear page per service. Put a plain answer near the top: what you do, who it's for, and where you work.
- Headings that sound like real questions. "How much does a drain cleaning cost?" is easier for AI to match than "Our Solutions."
- Schema markup. A small block of code that tells machines your business name, address, phone, hours and services. Most website builders have a setting or plugin for it.
- The same name, address and phone everywhere. Your site, your Google Business Profile and directories should all match, word for word.
- An llms.txt file. A short, plain-text map of your site, written for AI, at yourwebsite.com/llms.txt. It's a newer idea and not every AI tool reads it yet, but it takes 10 minutes. There's a sample below.
SampleWhat an llms.txt file looks like
Swap in your own business, then save it as llms.txt at the top level of your website.
# Your Business Name
> Plumbing in Long Beach, CA. Open 7 days. Call (555) 555-5555.
## Services
- [Drain cleaning](https://yourwebsite.com/drains): clogs, slow drains, camera checks
- [Water heaters](https://yourwebsite.com/water-heaters): repair and replacement
## About
- [Reviews](https://yourwebsite.com/reviews)
- [Contact and booking](https://yourwebsite.com/contact)
Want a free look?I'll check your site for you
Send me 3 things: your business, your website, and the one thing you most want more of (calls, sales or trust). I'll look and get back to you.
- This one isn't a tool, and there's no affiliate link on this page.
- Fixing these helps AI read your site. It doesn't guarantee AI will recommend you.